Side note - when I was chiming in on github and actually, I think, triggered gg to start merging this back in, I remember I was doing 32B_Q8 w/ 7B 4KL draft but I think I still had mine set to --draft 5; I will say 7B>>>>3B so far. I may have to play around with using some even smaller drafts
March 11, 2025 at 1:10 PM
Side note - when I was chiming in on github and actually, I think, triggered gg to start merging this back in, I remember I was doing 32B_Q8 w/ 7B 4KL draft but I think I still had mine set to --draft 5; I will say 7B>>>>3B so far. I may have to play around with using some even smaller drafts
32B Coder-Q8 w/ and w/out 7B-Q4_K_L draft - PSA speculative decoding is in llamacpp and works. (depending on your hardware, experiment w/ diff model sizes - ymmv vary wildly)
March 11, 2025 at 12:50 PM
32B Coder-Q8 w/ and w/out 7B-Q4_K_L draft - PSA speculative decoding is in llamacpp and works. (depending on your hardware, experiment w/ diff model sizes - ymmv vary wildly)
ok. told claude 3.7 to extra some settings mgmt components from an app layer into generic fastapi router+react component. I expected that to work. Him writing this beautiful readme with emojis I did *NOT* expect.
February 25, 2025 at 12:08 AM
ok. told claude 3.7 to extra some settings mgmt components from an app layer into generic fastapi router+react component. I expected that to work. Him writing this beautiful readme with emojis I did *NOT* expect.
then this was a ~easy question. Which makes it a bad problem. Whether you could norm your test in such a way that it didn't look bad it was still definitely bad, because it was too domain/cultural specific.
Which is interesting because leetcode is a shibboleth. Don't trust me.
January 3, 2025 at 1:05 PM
then this was a ~easy question. Which makes it a bad problem. Whether you could norm your test in such a way that it didn't look bad it was still definitely bad, because it was too domain/cultural specific.
Which is interesting because leetcode is a shibboleth. Don't trust me.
** I could have said "LLMs" or even "AI" but I don't want to get too bogged down here in the implementation. The synthesis of application of knowledge across use cases. Side note: I told o1-pro to pro/con debate my statement. ;)
December 31, 2024 at 1:49 PM
** I could have said "LLMs" or even "AI" but I don't want to get too bogged down here in the implementation. The synthesis of application of knowledge across use cases. Side note: I told o1-pro to pro/con debate my statement. ;)
omg I am dying. Claude takes "output as bash" as "pretend to be my bash shell" instead of "output as bash script" 😂 I'm going to take it home and feed it and keep it forever!
December 30, 2024 at 1:24 PM
omg I am dying. Claude takes "output as bash" as "pretend to be my bash shell" instead of "output as bash script" 😂 I'm going to take it home and feed it and keep it forever!
After downloading like 60+ artifacts click-to-download, using github.com/m9e/aishell to move them all to the right place after "head -5 *.*" to put the target into the llm context. AIshell is a toy but sometimes hilariously useful
December 8, 2024 at 12:21 AM
After downloading like 60+ artifacts click-to-download, using github.com/m9e/aishell to move them all to the right place after "head -5 *.*" to put the target into the llm context. AIshell is a toy but sometimes hilariously useful