flowersteam / explauto Goto Github PK
View Code? Open in Web Editor NEWAn autonomous exploration library
Home Page: http://flowersteam.github.io/explauto
License: GNU General Public License v3.0
An autonomous exploration library
Home Page: http://flowersteam.github.io/explauto
License: GNU General Public License v3.0
Should we put explauto/sensorimotor_model/forward and inverse in explauto/models ?
Pro: models should be in /models
Con: forward and inverse models are only used in sensorimotor_model
Hi,
Right after a fresh install of explauto, running:
python -c "import explauto"
raises the following error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "explauto/__init__.py", line 7, in <module>
from .sensorimotor_model.sensorimotor_model import SensorimotorModel
File "explauto/sensorimotor_model/__init__.py", line 7, in <module>
module = importlib.import_module('explauto.sensorimotor_model.{}'.format(mod_name))
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "explauto/sensorimotor_model/non_parametric.py", line 5, in <module>
from ..third_party.models.models.learner import Learner
ImportError: No module named third_party.models.models.learner
From what I remember, we removed the third_party
package more than a year ago. However, it seems that it is still used in non_parametric.py
(in particular the Learner
class).
@pierre-rouanet: do you remember the status of this?
@sebastien-forestier: do you still use the non_parametric
module? (e.g. for your LWR+CMAES inverse model?). Or do you rely on another module for LWR?
@jypuigbo: as a temporary fix you can:
File "explauto/sensorimotor_model/non_parametric.py", line 5, in <module>
from ..third_party.models.models.learner import Learner
and reinstall again. The error should disappear (you won't be able to use the non_parametric
module though. You can also check with @santmarti who recently installed it without mentioning this problem.
Cheers
Remove logs into np arrays
Adapt signature to be called by Agent -> compute x_card from bounds
Arguments can't be understood before an introduction to the general explauto architecture
@clement-moulin-frier: Looking into gmm_progress, I am wondering what the n_sample parameter stand for and what is a reasonable value for it.
https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/gmm_progress.py#L38L46
From what I understood it is the size of the window used, that is only the last n_sample are considered when fitting the GMM. Hence the self.data[self.t % self.n_samples, ...
with the % used as a rolling index?
The condition to trigger the update rule is also a bit hard to follow:
if abs(self.t % (self.n_samples * self.scale_t / 4.)) < self.scale_t:
self.update_gmm()
I think it updates every n_samples/4 steps, because scale_t is always 1.
Does this mean the first updates are done while self.data is not fully filled yet, hence full of zero? What is the potential effect of this on the algorithm, I guess it will just put a gaussian at [0,0,...,0] and maybe not find it 'interresting'.
(See https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/gmm_progress.py#L24: self.data = numpy.zeros((n_samples, len(expl_dims) + 2))
)
I migth propose a simplification for this and add the update frequency in the parameters.
I have met @sebastien-forestier last week and we have a proposition of architecture for the development of GMR. As it has been said before, it should be possible to use GMR in any case. That is why a GMR class will be created in models and will be herited from sklearn.mixture.GMM like GMM class from gmminf.py. Plus, it will contained regressions methods LSE, SLSE, stochastic sampling and the jonathan sampling method that has been described by @oudeyer inspired from @jgrizou noteboook . The number of gaussians are given in parameters.
NB :
A second class ILOGMR will be implemented and would be herited from SensorimotorModels abstract class . It will be available for inverse and forward model. Plus, users will choose the way they want to compute their inverse model (directly or using optimization).
Do you think this structure is fine ? Do I forget something ?
Because of the boundary constraints on the motor/sensory dimension, the sampling of a new motor/sensory command/goal is biased with increase probability to sample a point at the boundary.
This is due to line https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/gmm_progress.py#L34L35 where the sampled point is constrained to be within the boudaries. A quick visualization of the problem is below.
It leads to the following patterns of goal selection in a 7 arm experiments. The right plot show in red the goal selected, we clearly see an oversampling on the boundaries, here [-1, 1] in each dimension.
The simplest solution to unbias this sampling step is to keep sampling until the point sampled is within the boundary. The resulting resampling strategy looks much better/less biased. It was run with the same seed, the effect is the same for many seeds.
This has been implemented in pull request #79 by:
It seems rather complex to make all environment works with processes. For instance, the pypot environment doesn't work.
One option would be to choose when running an ExperimentPool if you want to use Thread or Process.
Maybe the environment (models ?) should themselves declare if they support processes or not.
Hi guys,
If I want to run an experiment over a few days, is there a way to stop, save, reload, and start again an experiment?
I found fast_forward(self, log)
in experiment.py
, it resets an experiment state given a log. Could you confirm this is all we need and thus the way to go?
If yes, I will implement a save and load function in the log class. What would be the best way in your opinion (pickle.. ) ?
Just looked into the competence_dist function: https://github.com/flowersteam/explauto/blob/master/explauto/interest_model/competences.py#L4
def competence_dist(target, reached, dist_min=0., dist_max=1.):
return (min(- dist_min, - np.linalg.norm(target - reached))) / dist_max
I don't get this division by dist_max, what is it meant for? Also why is the division not casted in float?
It results in very weird behavior like:
competence_dist(0,0,2,10.0) -> -0.2 (expected -2 ?)
competence_dist(0,0,2,10) -> -1 (expected -2 ?)
competence_dist(0,100,0,1) -> -100 (expected -1 ?)
I would have assumed that we rather would like to bound the error between dist_min and dist_max, isn't it? But I might be missing something deeper here.
The patch would be:
def competence_dist(target, reached, dist_min=0., dist_max=1.):
return max((min(- dist_min, - np.linalg.norm(target - reached))), -dist_max)
I also assume the move to negative domain is because competence should improve when error reduces.
Hi,
I am currently trying to run explauto on Poppy using your tutorial but I think it's not updated : poppy-software repository doesn't exist anymore. I think it has been replace by poppy-humanoid-beta so I cloned it and changed the config path '../../poppy-software/poppytools/configuration/poppy_config_linux.json' by '../poppy-humanoid-beta/software/poppytools/configuration/poppy_config.json'
But when I try to run
poppy = pypot.robot.from_json(config_path)
I got this
File "", line 3, in
poppy = pypot.robot.from_json(config_path)
File "/home/roxane/Logiciels/anaconda/lib/python2.7/site-packages/pypot/robot/config.py", line 244, in from_json
return from_config(config, sync=sync, strict=strict, use_dummy_io=use_dummy_io)
File "/home/roxane/Logiciels/anaconda/lib/python2.7/site-packages/pypot/robot/config.py", line 60, in from_config
dxl_io = dxl_io_from_confignode(config, c_params, attached_ids, strict)
File "/home/roxane/Logiciels/anaconda/lib/python2.7/site-packages/pypot/robot/config.py", line 139, in dxl_io_from_confignode
port = pypot.dynamixel.find_port(ids, strict)
File "/home/roxane/Logiciels/anaconda/lib/python2.7/site-packages/pypot/dynamixel/__init__.py", line 105, in find_port
ids, list(set(ids) - set(ids_founds))))
IndexError: No suitable port found for ids [31, 32, 33, 34, 35, 36, 37, 41, 42, 43, 44, 51, 52, 53, 54]. These ids are missing [32, 33, 34, 35, 36, 37, 41, 42, 43, 44, 51, 52, 53, 54, 31] !
I use spyder from anaconda to compile my script.
Do you know how to fix it ?
Thanks,
Roxane
I was wondering if the computation of progress in the discretized interest model is the intended one here.
Indeed, the np.cov is computing the covariance between [1, 2, 3, 4, 5] and the last competences e.g. [0.8, 0.6, 0.7, 0.9, 0.95].
An issue with that behavior is at the beginning of the exploration when few cells are sampled and few points are sampled in those cells.
The np.cov is in fact increasing even if progress is constant:
np.cov(range(3), [0.5, 0.6, 0.7])[0,1] = 0.099
np.cov(range(4), [0.5, 0.6, 0.7, 0.8])[0,1] = 0.166
np.cov(range(5), [0.5, 0.6, 0.7, 0.8, 0.9])[0,1] = 0.25
which leads to the exploration of n points in each randomly chosen cell with n corresponding roughly to the window's size.
That can be seen for instance in the notebook about curiosity in the scatter plots.
I tried another behavior in another branch here where the initial guessed progress of each unsampled cell is a constant (e.g. 10), and the progress is computed as the mean of the 5 last competencecs in the cell minus the mean of the 10 to 5 last ones.
In that case, there is an exploration biais that pushes to explore at least once in each cell before using progress based sampling.
What do you think ?
@clement-moulin-frier
@pierre-rouanet
Hi again,
I have been looking into sklearn recently and there is not gaussian mixture inference implemented. In explauto you implemented one, quite nicely by constructing on top of the GMM class from sklearn.
I think adding this to the official sklearn repo would be of tremendous help for many other people. Do you think the code is mature enough? Are there specific basic cases that are not handled? Is it worth our time?
In the same line, once gmminf is available, adding ilo_gmm shoudl be easy and could help diffusing this algortihm in the community.
in sensorimotor models there is a check for wether we do a forward or inverse prediction like
if in_dims == self.m_dims and out_dims == self.s_dims: # forward
elif in_dims == self.s_dims and out_dims == self.m_dims: # inverse
if s_dims and m_dims happen to be equal i guess this will fail?
I just tried to install explauto. To have import explauto
working, I had to also install matplotlib and pypot. Should we add those to the dependency list?
Tutorials:
J'aimerais pouvoir porter sur Python le code SGIM-ACTS (https://flowers.inria.fr/Nguyen-Oudeyer-Paladyn2013.pdf).
J'aurai quelques questions concernant votre librairie exploauto.
Avez-vous déjà fait des tests en utilisant votre librairie et un agent pouvant changer de stratégie ?
(choisir activement entre différentes stratégies comme imitation, random, SAGG-RIAC)
Si oui, comment avez-vous appliquer les interest trees pour qu'ils représentent différentes stratégies en même temps ?
J'ai pas l'impression que le code soit compatible avec des actions complexes (séquences de taille variable composées de primitives motrices), pouvez-vous confirmer ?
Comme je devrais à terme utiliser des actions complexes, me conseillez-vous du coup d'utiliser votre librairie pour cela ?
Avez-vous également fait des tests pour un agent pouvant changer activement de task space ?
Voilà, j'hésite à m'efforcer de comprendre votre librairie et faire quelque chose de compatible ou faire quelque chose d'entièrement différent.
Motor babbling is almost always better evaluate than goal babbling, whereas the latter covers the sensory space much better
Change the relationship between the number of nearest neighbors and the number of gaussian. The number of nearest_neighbors should be inffered from the number of gaussians (fixed to 2 or 3) and the dimensionality.
I am now looking into the testcases generation. I find the method to generate uniform testcases over the sensory space very appealing. Code is here: https://github.com/flowersteam/explauto/blob/master/explauto/environment/testcase.py
My understanding is that a grid of a given resolution is projected on the sensory space, and that each cell is associated with only one observation from within that cell. My questions concern the resolution parameter:
resolution = max(2, int((1.3*n)**(1.0/len(robot.s_feats))))
?I also noticed this: # TODO : change obs only if nearer from center of coo.
From what I understand is that in each cell, the corresponding observation will be the last observation encountered in the _populate process. The todo is to replace that by keeping the closest to the center of the cell?
Hi,
I have tried to install explauto from pip / easy_install and source but have this trouble ::
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'extra_require'
I have tried # pip install --upgrade setuptools
Requirement already up-to-date: setuptools in /usr/lib/python3.4/site-packages
Cleaning up...
but the trouble continue
Thanks for your help.
from a quick check it appears to me that explauto explicitly assumes a fixed starting state or, equivalently, kinematic motion?
explauto: m -> FWD -> s_{t+1}, s_{t+1} -> INV -> m
desired: (s_t, m) -> FWD -> s_{t+1}, (s_t, s_{t+1}) -> INV -> m) with (): vector concatenation
could this be done in a minimally invasive manner?
Me spamming again!
I looked quickly into gmminf.py
Are all the part of the code actually in use?
The conditional
functions seem not in use inside gmminf as well as not in use in ilogmm. There are the following:
def conditional(mean, covar, dims_in, dims_out, covariance_type='full'):
""" Return a function f such that f(x) = p(dims_out | dims_in = x) (f actually returns the mean and covariance of the conditional distribution
"""
in_in = covar[ix_(dims_in, dims_in)]
in_out = covar[ix_(dims_in, dims_out)]
out_in = covar[ix_(dims_out, dims_in)]
out_out = covar[ix_(dims_out, dims_out)]
in_in_inv = inv(in_in)
out_in_dot_in_in_inv = out_in.dot(in_in_inv)
cond_covar = out_out - out_in_dot_in_in_inv.dot(in_out)
cond_mean = lambda x: mean[dims_out] + out_in_dot_in_in_inv.dot(x - mean[dims_in])
return lambda x: [cond_mean(x), cond_covar]
and
def conditional(self, in_dims, out_dims):
conditionals = []
for k, (weight_k, mean_k, covar_k) in enumerate(self):
conditionals.append(conditional(mean_k, covar_k,
in_dims, out_dims,
self.covariance_type))
cond_weights = lambda v: [(weight_k * Gaussian(mean_k[in_dims].reshape(-1,),
covar_k[ix_(in_dims, in_dims)]).normal(v.reshape(-1,)))
for k, (weight_k, mean_k, covar_k) in enumerate(self)]
def res(v):
gmm = GMM(n_components=self.n_components,
covariance_type=self.covariance_type,
random_state=self.random_state, thresh=self.thresh,
min_covar=self.min_covar, n_iter=self.n_iter, n_init=self.n_init,
params=self.params, init_params=self.init_params)
gmm.weights_ = cond_weights(v)
means_covars = [f(v) for f in conditionals]
gmm.means_ = array([mc[0] for mc in means_covars]).reshape(self.n_components,
-1)
gmm._set_covars(array([mc[1] for mc in means_covars]))
return gmm
return res
self.in_dims = array(in_dims)
self.out_dims = array(out_dims)
means = zeros((self.n_components, len(out_dims)))
covars = zeros((self.n_components, len(out_dims), len(out_dims)))
weights = zeros((self.n_components,))
sig_in = []
inin_inv = []
out_in = []
mu_in = []
for k, (weight_k, mean_k, covar_k) in enumerate(self):
sig_in.append(covar_k[ix_(in_dims, in_dims)])
inin_inv.append(matrix(sig_in).I)
out_in.append(covar_k[ix_(out_dims, in_dims)])
mu_in.append(mean_k[in_dims].reshape(-1, 1))
means[k, :] = (mean_k[out_dims] +
(out_in *
inin_inv *
(value - mu_in)).T)
covars[k, :, :] = (covar_k[ix_(out_dims, out_dims)] -
out_in *
inin_inv *
covar_k[ix_(in_dims, out_dims)])
weights[k] = weight_k * Gaussian(mu_in.reshape(-1,),
sig_in).normal(value.reshape(-1,))
weights /= sum(weights)
def p(value):
# hard copy of useful matrices local to the function
pass
return p
The second one is specially cryptic to me, I don't understand what is going on there. Maybe those are archaeological remainings :D ?
Also warns (ValueError?) user when they want to use the same test cases with different environments.
Hi guys,
I started using ILOGMR to compare it with various regression algortihm in sci-kit learn. For this I had to create a class wrapping the ilo_gmm from explauto to an ILOGMR estimator in sklearn.
Should I had this file in the explauto repo? If yes, where should it land?
Currently the class is as follow:
import numpy
from sklearn.base import BaseEstimator
from explauto.sensorimotor_model import ilo_gmm
class ILOGMR(BaseEstimator):
def __init__(self, conf, n_components=3, n_neighbors=100, random_state=None):
self.conf = conf
self.explauto_ilo_gmm = ilo_gmm.IloGmm(conf, n_components)
self.explauto_ilo_gmm.n_neighbors = n_neighbors
self.random_state = random_state
def fit(self, X, y):
self.explauto_ilo_gmm.dataset.reset()
for n in range(X.shape[0]):
self.explauto_ilo_gmm.update(X[n, :], y[n, :])
def compute_conditional_gmm(self, x):
return self.explauto_ilo_gmm.compute_conditional_gmm(
self.conf.m_dims, self.conf.s_dims, x)
def predict(self, X):
y_pred = numpy.zeros((X.shape[0], len(self.explauto_ilo_gmm.s_dims)))
for n in range(X.shape[0]):
gmm = self.compute_conditional_gmm(X[n, :])
y_pred[n, :] = numpy.sum(gmm.means_.T * gmm.weights_)
return y_pred
This use the new interface for ilo_gmm.py that is currently under pull request #49
It still lacks the set_param, get_parm method so we can use it with the really convenient GridSearchCV but it will come at some point.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.