Changing weights 

Hello,

I am working on a project in framsticks and want to develop learning
creatures. The problem with this is that in any type of learning, especially
hebbian, there has to be an interaction between neurons to change the
weights. So my question is: is it possible to get a reference to an input
neuron AND change the weights from that neuron to your own. I tried looking
a the NeuroProperties object, but it has no documentation.

Regards,

Tom Schut

Forums: 
Maciej Komosinski's picture

Hi Tom,

> I am working on a project in framsticks and want to develop learning
> creatures. The problem with this is that in any type of learning, especially
> hebbian, there has to be an interaction between neurons to change the
> weights. So my question is: is it possible to get a reference to an input
> neuron AND change the weights from that neuron to your own. I tried looking
> a the NeuroProperties object, but it has no documentation.

Let me quote my answer to Matej from Prague who was also interested
in this issue:

> And I have one technical question. I would like to use different types of
> synapses, to allow learning (change of synapses, i.e. weights) in an
> individual (different types of Hebbian learning) and to evolve just
> parameters of the synapses (e.g. excitatory/inhibitory, method of
> learning...). Are such things theoretically possible (if I use the
> script), or the software allows only simple weights between neurons, that
> do not change in an individual?

Weights themselves cannot change (as encoded in the genotype), but
you can do virtually anything in a neuron that you define in the *.neuro
script file, i.e. you can define which neuron parametes whould be evolved
and which should not, and how these parameters affect neuron
characteristics. So you can easily overcome the constraint of constant
weights - for example, all weights could be always "1", and your
"dynamic" weights would be a property of a neuron.

> 1)How to make the neurons working? I can make my own neuron definition,
> where I can edit properties. The synapses can be at the target
> (postsynaptic) neuron. However, I need a synapse for every input neuron.
> So I need something like a dynamic array and I also need access to the
> input neurons (I have to know if it did not fire as well, if rule
> presynaptic). Or can I somehow control everything from the onStep) of the
> experiment definition?

You can have as many local variables/objects as you need in each
neuron. For example, you can create an array with the size equal
to the number of inputs in each neuron. *See the scripts/delay.neuro
file*.

You can get (unweighted) inputs using the getInputState function.

> 2) I hope I studied all available documentation, but I am missing some
> description of relations between the classes, what is loaded when and
> where from... Is there perhaps a documentation for programmers? I have
> read the Scripting presentations, but that does not really uncovers the
> data flow between the objects... For instance I am not sure where Neuro
> and NeuroClass live and how can I get to them... I would also like to
> know, how the activations spread in the nn, is it synchronous or
> asynchronous...

It is synchronous. Each neuron sets its own output state.

As to the data flow... in Framsticks, you make event handlers
or dynamically create objects, or use "static" objects. Some
objects only exist (are accessible) in some specific contexts,
shown as filled rectangles in the left frame at
http://www.framsticks.com/common/script/docs/index.html

> 3)Do my properties of synpases get automatically evolved under f1, if
> added to neuron definition (property:)? How will they be encoded,
crossed-over
> etc.? Or do I have to create my own encoding?

Yes; simple, published (i.e. not those with flags:32) properties are
automatically mutated (simple are integer or floating point properties).
String properties or properties of other types are not mutated.

MacKo

Thanks, I will try to do it in this way.

I still have one question left.

I'm trying to make a creature that behaves in a biological way by perceiving
its own internal state like being fertile or hungry.
Now I'm not sure about how to model this. I could make a sensor connected to
the neural network that simply sets its state to the fertility variable or I
could add the vertility variable to the getInputState of each neuron since
this is an internal variable that needs no perceiving.

This is in essence of course the same approach, but I was wondering which
approach is computationally more attractive. A third, but not biological
solution is to hard-code the rules like if(fertile) than search mate.

Regards,

Tom Schut

"Maciej Komosinski"
schreef in bericht news:e84ddf$sd5$1@cancer.cs.put.poznan.pl...
> Hi Tom,
>
>> I am working on a project in framsticks and want to develop learning
>> creatures. The problem with this is that in any type of learning,
>> especially hebbian, there has to be an interaction between neurons to
>> change the weights. So my question is: is it possible to get a reference
>> to an input neuron AND change the weights from that neuron to your own. I
>> tried looking a the NeuroProperties object, but it has no documentation.
>
> Let me quote my answer to Matej from Prague who was also interested
> in this issue:
>
>
> > And I have one technical question. I would like to use different types
> > of
> > synapses, to allow learning (change of synapses, i.e. weights) in an
> > individual (different types of Hebbian learning) and to evolve just
> > parameters of the synapses (e.g. excitatory/inhibitory, method of
> > learning...). Are such things theoretically possible (if I use the
> > script), or the software allows only simple weights between neurons,
> > that
> > do not change in an individual?
>
>
> Weights themselves cannot change (as encoded in the genotype), but
> you can do virtually anything in a neuron that you define in the *.neuro
> script file, i.e. you can define which neuron parametes whould be evolved
> and which should not, and how these parameters affect neuron
> characteristics. So you can easily overcome the constraint of constant
> weights - for example, all weights could be always "1", and your
> "dynamic" weights would be a property of a neuron.
>
>
> > 1)How to make the neurons working? I can make my own neuron definition,
> > where I can edit properties. The synapses can be at the target
> > (postsynaptic) neuron. However, I need a synapse for every input neuron.
> > So I need something like a dynamic array and I also need access to the
> > input neurons (I have to know if it did not fire as well, if rule
> > presynaptic). Or can I somehow control everything from the onStep) of
> > the
> > experiment definition?
>
> You can have as many local variables/objects as you need in each
> neuron. For example, you can create an array with the size equal
> to the number of inputs in each neuron. *See the scripts/delay.neuro
> file*.
>
> You can get (unweighted) inputs using the getInputState function.
>
> > 2) I hope I studied all available documentation, but I am missing some
> > description of relations between the classes, what is loaded when and
> > where from... Is there perhaps a documentation for programmers? I have
> > read the Scripting presentations, but that does not really uncovers the
> > data flow between the objects... For instance I am not sure where Neuro
> > and NeuroClass live and how can I get to them... I would also like to
> > know, how the activations spread in the nn, is it synchronous or
> > asynchronous...
>
> It is synchronous. Each neuron sets its own output state.
>
> As to the data flow... in Framsticks, you make event handlers
> or dynamically create objects, or use "static" objects. Some
> objects only exist (are accessible) in some specific contexts,
> shown as filled rectangles in the left frame at
> http://www.framsticks.com/common/script/docs/index.html
>
> > 3)Do my properties of synpases get automatically evolved under f1, if
> > added to neuron definition (property:)? How will they be encoded,
> crossed-over
> > etc.? Or do I have to create my own encoding?
>
> Yes; simple, published (i.e. not those with flags:32) properties are
> automatically mutated (simple are integer or floating point properties).
> String properties or properties of other types are not mutated.
>
>
> MacKo

Maciej Komosinski's picture

Hi,

> I'm trying to make a creature that behaves in a biological way by perceiving
> its own internal state like being fertile or hungry.
> Now I'm not sure about how to model this. I could make a sensor connected to
> the neural network that simply sets its state to the fertility variable or I
> could add the vertility variable to the getInputState of each neuron since
> this is an internal variable that needs no perceiving.

What about simply using the Energy sensor?

MacKo

Oops, forgot about that one, I should read the manual more carefully. I have
one final question. I want to run my simulations on a macOS network, and
even though mac runs on a freebsd kernel the unix compilation for the CLI
doesn't work on mac. Is there a version I could use or compile on mac?

Tom Schut
"Maciej Komosinski"
schreef in bericht news:e8c54v$7a6$1@cancer.cs.put.poznan.pl...
> Hi,
>
>> I'm trying to make a creature that behaves in a biological way by
>> perceiving its own internal state like being fertile or hungry.
>> Now I'm not sure about how to model this. I could make a sensor connected
>> to the neural network that simply sets its state to the fertility
>> variable or I could add the vertility variable to the getInputState of
>> each neuron since this is an internal variable that needs no perceiving.
>
> What about simply using the Energy sensor?
>
>
> MacKo

Maciej Komosinski's picture

> Oops, forgot about that one, I should read the manual more carefully. I have
> one final question. I want to run my simulations on a macOS network, and
> even though mac runs on a freebsd kernel the unix compilation for the CLI
> doesn't work on mac. Is there a version I could use or compile on mac?

Hmm, there is no official compilation for Mac. But I suppose there is
some emulation on Mac for either DOS/Win32 commandline or Linux
commandline?

Mac