From what I understand, Gene Regulatory Network [1] works very differently from Biological Neural Circuits [2]. Finally, Perceptron used in computers [3] is based on arithmetic operations and is again different from any of those two biological mechanisms. However, perceptron appears to be a good tool to model functions that depends on a large number of parameters.
What we may infer from that is that after a lot of time of evolution, functions depending on a lot of parameters appear. Yet Biological Neural Circuits seem quite bad at approximating simple continuous functions such as the multiplication between two numbers.
Gene Neural Network and Biological Neural Circuit are quite impressive structures considering their size, materials, and energy constraints. However if you allow bigger size, more energy, and faster conducting materials, it should be possible to do have faster modeling tools.
Moreover, to better take into account non-linear functions, my two cents is that Perceptrons could be further enhanced using piecewise polynomial functions instead of piecewise linear functions [4,5].
To me it seems like the fundamental idea is still the same. You have nodes that are connected to each other with strengths that can change according to feedback and complicated logic emerges from that.
That seems like almost fundamental emergent idea that is behind all the intelligence in the World.
It seems to allow for intelligence to occur, without this type of concept everything would just be random and chaotic.
The point is, how amazing it is that complicated behaviour can arise from this simple idea.
At this point it seems, that neural networks should really be taught at schools as soon as possible.
> That seems like almost fundamental emergent idea that is behind all the intelligence in the World.
That is indeed quite nice, yet intelligence seems much more diverse in biological organisms and not always reduced to nodes and edge strengths. The nodes and edges are a way to store data and to modify it that seems particularly well adapted mechanisms based on electricity.
On the other hand, there exists other very different (and slower) biological intelligent mechanisms that are not based on electricity. For example, a tree is perfectly adapted to its environment, capable of taking energy from the sun, materials from the ground, transform it, and so on. Yet the intelligent mechanism that created trees is not based on neural networks as far as I understand it. In this mechanism, the data is stored in DNA, and DNA can be rewriten at each new generation. It is a completely different (and slower) approach.
Finally I agree that the emergence of fast intelligence through neural network is incredible, although for me the first impressive advance is the use of the electricity to speed up the information exchange. The arrangements in nodes and edges could follow naturally from the fact that the transmission of electric signals works better through 1D circuits. The second impressive advance is the way the networks are layered. This is critical and very difficult to have an arrangement of edges and nodes that can both be trained and inferred efficiently. Without the proper arrangement, a biological neural network or a computer circuit is much less intelligent.
Finally in the future we could imagine intelligent mechanisms based on quantum mechanics for example. In this case, it would probably be vastly different from nodes and edges, due to the underlying physical constraints that are different in electricity and in quantum mechanics.
tldr: yes nodes and edges seem fundamental in fast intelligence based on electricity, but other very different intelligent mechanism exist in biological organisms, and other very different intelligent mechanisms could still be invented in the future.
What we may infer from that is that after a lot of time of evolution, functions depending on a lot of parameters appear. Yet Biological Neural Circuits seem quite bad at approximating simple continuous functions such as the multiplication between two numbers.
Gene Neural Network and Biological Neural Circuit are quite impressive structures considering their size, materials, and energy constraints. However if you allow bigger size, more energy, and faster conducting materials, it should be possible to do have faster modeling tools.
Moreover, to better take into account non-linear functions, my two cents is that Perceptrons could be further enhanced using piecewise polynomial functions instead of piecewise linear functions [4,5].
[1]: https://en.wikipedia.org/wiki/Gene_regulatory_network
[2]: https://en.wikipedia.org/wiki/Neural_circuit
[3]: https://en.wikipedia.org/wiki/Perceptron
[4]: https://doi.org/10.1109/TPAMI.2021.3058891
[5]: https://doi.org/10.1109/TPAMI.2022.3231971