based on what it knows about programms that supposedly output fibonnaci sequences

In contrast look at GPT4 - it literally seems to model the entire program and run it

Here's a really simple example of what I mean. Look at the GPT 3 output for running this program - it infers the program is a fibonnaci program and knows that fibonacci programs output a fibonnaci sequence. it's simulating the entire program

I'm definitely curious as well - generally have wondered what the best way to test this sort of thing is - I wouldn't know how to really test even if I had it

Yes - this is what I've done previously, but with 4 it's much less necessary, if at all

Instead language models should be created that have sliding context windows, and within some context width execute linguistic instructions

It’s not really correct to think “gpt will just do everything” - there’s always going to be a huge specification problem and programming will still exist in some sense for the construction of experiences

The biggest change from gpt 3->4 is the complexity and number of steps that 4 correctly obeys during execution - davinci2 in principle could execute the same linguistic programs as 4, but with multiple linguistic programs interspersed with JavaScript