also, i am implementing a sort of... approximation? of what backpropagation does for deep neural networks, but this system isnt differentiable in the first place, so it is merely a dim shadow of what you might expect from backprop. honestly calling it a neural network at all is a bit of a stretch...

ill probably keep trying to train it on image classification for the time being, but i dont expect great results from this. next step for Silky would be to have it play a more complex game like tetris or minesweeper or something. something simple that can run well enough on my home server's 1070 lol

it also doesnt help that this kind of training basically does away with the primary strength of a CNN - its memory. i have to reset its variable parameters related to its memory each time i want to expose it to a new image to classify. it kind of defeats the purpose of even using a CNN at all.

Its almost like the exposure time of a camera... you have to sit there and wait for it to think about it before you can hope to get a single answer from it. this ends up slowing down its processing of training data by a significant margin.

In order to get it to get a single output from it that might be an answer to "which image classification is this?" when showing it something from ImageNet, I found I need to give the network an "exposure time" to the image before it can give me any sort of coherent output back.

Some of the design choices that were required for creating a cognitive neural network, like variating parameters and stepwise updates of parameters, mean that this kind of NN thrives in a game environment, but really suffers at answering the kind of "yes/no" questions ImageNet asks of it.

So I started trying to train my model on ImageNet, just to see how it would do. I'm not done setting it up yet, but one thing I've noticed already is that this type of "cognitive" NN design actually is very bad at training on large datasets... by, well, design.

im not exactly sure what dataset to use, though? maybe a benchmark dataset like imagenet would be good. i dont like how imagenet got its data, it feels icky to use it, but it seems like thats an industry standard benchmark so maybe i should anyway...

but anyway, next step for this project is to get serious about how to train the model. i need to develop a much more rigorous training algorithm and use an actual dataset for training, rather than a simple game.

with the default settings, both the models in Silky should run on less than 8gigs of vram! so if you have a pc with a modern graphics card, odds are you should be able to start training a model.

big news though! it seems like ive gotten through the bugs in a new model in Silky, should be usable. if you'd like to playtest this silly project, just download the .whl from the latest release, import it to a python venv, and run the "run_velvet()" function from the "iteration" module.

so i guess ill just blog a bit about my ML project here... maybe someone will find it interesting? i'm an amateur in this field so there's a lot of stuff i dont have a complete understanding of, maybe if you see something i misunderstand let me know, yea?

when the default html page loads in (it took .000347s and you weren't prepared)

hi bsky! im a swe making an open source ML project called Silky. its a python library built on Torch that offers a new design for neural networks - one that lets NNs have persistent memory and act in real time. i'd love it if you checked it out! #programming #swe #ML #AI
github.com/octakitten/s...