👀

I think having universities pay PhD students would also give them more academic freedom. In CS labs, for instance, it’s common for the PI to get funding which is directed toward a particular project and PhD students are just getting paid to do these projects. That’s not always leading to new ideas

To be sure, I believe in the community, I just think this is a big shift relative to just a couple of years ago when companies only had a compute moat. Now they seem to have both a compute and method moat. N/N

I think another interesting point here is how far industry has gotten ahead of open source/academia on building these systems. This was a whole talk trying to figure out how to reinvent the wheel. 7/N

Also, in case the bitter lesson wasn't enough, of course Rich Sutton had something to say about self-verifying AI in 2001! incompleteideas.net/IncIdeas/Key... 6/N

For example, should we ever expect an LLM to weigh in on whether P=NP? There are also more subjective domains requiring a notion of correctness, such as navigating interpersonal relations, where you can't write unit tests for every scenario you'll encounter. 5/N

For instance, if we build verifiers that are capable of verifying the math and science we currently know, will it be able to push on and generate and verify new solutions to open problems? If not, then my question is, how do we build verification systems that are not bounded by human feedback? 4/N

It seems like the system as a whole might still be bounded by the verification model's capabilities and how much they want to pay human annotators. 3/N

I wonder if this 2-model system (primary model + verifier model) will be enough to reach escape velocity (i.e., primary models that can generate new ideas or reason about ideas they haven't been exposed to before). 2/N

What about private GitHub repos? Do you think info contained in those are already being used or will be used to train models?