RAPID-MIX_API issueshttps://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues2018-01-19T14:25:20Zhttps://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/54API methods for training data quantification2018-01-19T14:25:20ZFrancisco BernardoAPI methods for training data quantificationThe quantity of data should be made visible on the high level interfaces — quantity of data can have multiple aspects to it — (e.g., #recorded Rounds, #examples per round, #total memory consumption). In one instance, one participant no...The quantity of data should be made visible on the high level interfaces — quantity of data can have multiple aspects to it — (e.g., #recorded Rounds, #examples per round, #total memory consumption). In one instance, one participant noticed the memory of the browser was reaching 3GB, and found this unusual, and also observed the consequences in training time.
Recommendation: Participants would benefit from API methods (namely in the training data class) that provide metrics about quantity or volume of data. This should help developers in design by making them explicit through the visual interface metaphors they develop. It should also be explained in the documentation the consequences of dealing with large data sets (e.g., training time, impact on the classification results outcomes, etc.)https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/36Writing examples as building blocks2017-09-20T14:03:39ZFrancisco BernardoWriting examples as building blocksExamples were considered “cool", “ea. one useful in its one way", “the whole set providing of building blocks” which can be “borrowed”.
Recommendation:
* Users will benefit from examples become building blocks that they can readily in...Examples were considered “cool", “ea. one useful in its one way", “the whole set providing of building blocks” which can be “borrowed”.
Recommendation:
* Users will benefit from examples become building blocks that they can readily integrate in their code (scaffolding for capturing data, either to local storage or to a service)
* Some of the external libraries like the MYO have hidden global values it makes it difficult to use. The building blocks (i.e., sensor bridges) should have the exposed methods and state carefully designed, just as the API.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/35Make server-side technologies robust and resilient, and pack them for local i...2017-12-02T23:09:29ZFrancisco BernardoMake server-side technologies robust and resilient, and pack them for local installationAll technologies should be made cross-platform (windows & linux support), to prevent the risk of the RAPID-MIX API to miss out on a significant share of end-users. This happened recurrently across the many of the UCD sessions.
Recommend...All technologies should be made cross-platform (windows & linux support), to prevent the risk of the RAPID-MIX API to miss out on a significant share of end-users. This happened recurrently across the many of the UCD sessions.
Recommendation: Server-side technologies should be open-sourced, and made robust and resilient for quick set and local installation, specially during UCD and dissemination actions.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/34Communicating the Interactive Machine Learning workflow to users2017-09-20T13:35:28ZFrancisco BernardoCommunicating the Interactive Machine Learning workflow to usersThe interactive machine learning workflow is difficult to convey with static sources of descriptive knowledge, such as text (documentation, comments, etc.) and even code. This is because of of procedural nature that the workflow entails....The interactive machine learning workflow is difficult to convey with static sources of descriptive knowledge, such as text (documentation, comments, etc.) and even code. This is because of of procedural nature that the workflow entails. Live demonstrations have been working pretty well for us, but were not always there.
Recommendation: We should find way to complement the documentation with richer media. Richer media, such as video and animations, are more effective at communicating the procedural nature of the workflow and yield better adoption.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/33API documentation fragmentation2017-09-20T13:04:40ZFrancisco BernardoAPI documentation fragmentationUsers' feedback reports about how the documentation of the different components is fragmented: different sites; different content, design and user experiences, and potentially, different authors.
Recommendations:
* There should be a ...Users' feedback reports about how the documentation of the different components is fragmented: different sites; different content, design and user experiences, and potentially, different authors.
Recommendations:
* There should be a common medium, method and style for documentation, which will reduce the adoption friction.
* ORM for MySQL, Postgres, loadash, underscore, KNX, and Processing, were given as reference examples of good documentation.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/32Explain the relation between inputs, classes, outputs and mapping2017-09-20T12:47:14ZFrancisco BernardoExplain the relation between inputs, classes, outputs and mappingOn the same note, how the relation between how inputs (or features) and outputs (or labels) is set was cumbersome. Some participants tried to train the same model with a different number of inputs.
Recommendation: In the case of compos...On the same note, how the relation between how inputs (or features) and outputs (or labels) is set was cumbersome. Some participants tried to train the same model with a different number of inputs.
Recommendation: In the case of composite labels (e.g. multiple regression outputs associated to the parameters of a synthesiser), it would be helpful to explain incrementally the rationale behind this, specially given this is one element that originates from our domain of practice and that makes our API highly opinionated.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/31Contrasting examples2017-09-20T12:40:44ZFrancisco BernardoContrasting examplesWe have insufficient examples that contrast how to use different machine learning tasks (classification vs regression).
Recommendation: Create examples that demonstrate how to use classification AND regression, or classification AND DTW"We have insufficient examples that contrast how to use different machine learning tasks (classification vs regression).
Recommendation: Create examples that demonstrate how to use classification AND regression, or classification AND DTW"https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/30Design patterns for using the RAPID-MIX API2017-09-20T12:59:27ZFrancisco BernardoDesign patterns for using the RAPID-MIX APIRAPID-MIX API users, as developers, would benefit from design patterns for macro or frequent use operations, that should be consistently implemented across examples.
Recommendations:
* Users would benefit from having the "record"...RAPID-MIX API users, as developers, would benefit from design patterns for macro or frequent use operations, that should be consistently implemented across examples.
Recommendations:
* Users would benefit from having the "record", "train" and "run" controls and modes made as visible and explicit as possible. A design pattern that unifies this kind of usage across examples, would provide a stronger scaffolding for understanding the RAPID-MIX workflow.
* Participants didn’t explore much feature selection in their process, sticking to what was given by default. Participants would benefit from a design pattern, or any scaffolding, that structures feature selection and the features meta-information across the examples.
* There aren’t many examples that save/load the “configuration data” (training data set or model). It would be useful to provide a design pattern for API examples that offers selection between pre-trained models (inputing stored training data, either from local or remote storage) and a new model to be trained from scratch, so that they can explore other features of the pipeline working with data, a feature selection and ML algorithms and parameters tuning.
* A design pattern that makes explicit the fast, realtime, and iterative nature of data streams, could benefit the users in understanding how to design with data. This could be achieved by rendering the feature vector or frame of data that is passed as input for training
https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/28Self-explanatory code and commenting practices2017-09-20T12:31:53ZFrancisco BernardoSelf-explanatory code and commenting practicesUsers considered code comments as important as documentation. Code comments compete with the main documentation, particularly for certain users who bypassed documentation completely, or who prefer to refer to it much later in the explora...Users considered code comments as important as documentation. Code comments compete with the main documentation, particularly for certain users who bypassed documentation completely, or who prefer to refer to it much later in the exploration and development workflow, for troubleshooting for instance.
Recommendations:
* Examples' code should be comprehensively commented in sections with “magic numbers” (e.g. number 5 in RapidStream, number of Gaussians, etc.)"
* Examples such as the RapidStream would benefit from adding the rationale and usefulness for each feature in the UI.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/26Providing an incremental approach for learning how to use the API2017-09-20T12:16:21ZFrancisco BernardoProviding an incremental approach for learning how to use the APIUsers suggested of structuring documentation with an "incremental approach", as "recipes", or short stories. Nature of Code was reference of providing a good incremental approach.
Recommendation: Provide guides grounded on specific u...Users suggested of structuring documentation with an "incremental approach", as "recipes", or short stories. Nature of Code was reference of providing a good incremental approach.
Recommendation: Provide guides grounded on specific use cases that show how to implement an incremental approach. For instance, the mouse examples are good candidates to illustrate how to start how to evolve from a barebones example with pre-defined data, to using real-time data, to mixing classification and regression, through to building up a more compelling use case grounded on real world scenario.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/25Provide further resources for the different underlying algorithms2017-09-20T12:10:27ZFrancisco BernardoProvide further resources for the different underlying algorithmsThe conceptual part about algorithms was considered both important and lacking. Users asked for more insights about how the algorithms worked.
Recommendation: Provide a succinct explanation with links to further resources, links to ext...The conceptual part about algorithms was considered both important and lacking. Users asked for more insights about how the algorithms worked.
Recommendation: Provide a succinct explanation with links to further resources, links to external 3rd party resources, such as videos, papers, etchttps://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/24Introduce supporting technologies (git, terminal, node, etc) or link to curat...2017-09-20T12:00:55ZFrancisco BernardoIntroduce supporting technologies (git, terminal, node, etc) or link to curated 3rd party resourcesSupporting technologies such as git, terminal, node, server-side technologies should be explained better.
Recommendation: Either with a very succinct explanation about rational and procedure, and/or a link to a good external resource th...Supporting technologies such as git, terminal, node, server-side technologies should be explained better.
Recommendation: Either with a very succinct explanation about rational and procedure, and/or a link to a good external resource that is compatible with our and language style.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/23Providing the big picture of ML concepts and how they relate to our API2017-09-21T13:28:11ZFrancisco BernardoProviding the big picture of ML concepts and how they relate to our APIThere is no element in the documentation that provides a broad, encompassing and big picture of ML concepts and how our API provides them. Users at eNTERFACE17 suggested some useful artefacts for help with this, such as mind maps or info...There is no element in the documentation that provides a broad, encompassing and big picture of ML concepts and how our API provides them. Users at eNTERFACE17 suggested some useful artefacts for help with this, such as mind maps or infographics.
This should include terms such as "features", "feature vector", "model", "classifier", "instance", "example", "label", etc.
Recommendation: Provide a mind map embedded in a webpage of the RAPID-MIX API, which relate fundamental ML concepts to what our API offers to end users. There is a glossary of terms by Ron Kohavi (http://robotics.stanford.edu/~ronnyk/glossary.html) that could be mapped to this React MindMap component (https://github.com/learn-anything/react-mindmap). Checkout this example (https://learn-anything.xyz/machine-learning/machine-learning-libraries)https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/21Regression requires additional explanation on the website2017-09-21T13:30:55ZFrancisco BernardoRegression requires additional explanation on the websiteRegression requires additional explanation, mainly with regards to :
* why is it different from interpolation
* why it doesn’t converge to the initial values
* ...Regression requires additional explanation, mainly with regards to :
* why is it different from interpolation
* why it doesn’t converge to the initial values
* ...https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/18"Hello World" is lame2017-12-04T16:14:23ZMichael Zbyszyński"Hello World" is lameThe current "hello world" example on the website is not very inspiring. It demonstrates something that is music-centred and esoteric. And, it doesn't work very well. Can this be improved? Or, should we abandon "hello world"?
One big pr...The current "hello world" example on the website is not very inspiring. It demonstrates something that is music-centred and esoteric. And, it doesn't work very well. Can this be improved? Or, should we abandon "hello world"?
One big problem is that this example should run without any 3rd party dependancies. Unfortunately, that means very limited input potential.
Also, how inspiring is "hello world" supposed to be?Michael ZbyszyńskiMichael Zbyszyńskihttps://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/16Stackoverflow presence2017-09-21T16:16:08ZFrancisco BernardoStackoverflow presenceFrancisco BernardoFrancisco Bernardohttps://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/4Document background classes2017-09-29T09:22:11ZMichael ZbyszyńskiDocument background classesUsers of classifiers (DTW or kNN) want to know when there is no gesture that matches what has been trained. We need to document how the possible use of a "background" class, or rejecting bad matches by thresholding.Users of classifiers (DTW or kNN) want to know when there is no gesture that matches what has been trained. We need to document how the possible use of a "background" class, or rejecting bad matches by thresholding.https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/38Reconsider trainingData.startRecording()2017-11-27T17:59:51ZMichael ZbyszyńskiReconsider trainingData.startRecording()@fabian Brought up a point about the trainingData class, and its current, stateful API.
It does this:
```cpp
trainingData.startRecording("myLabel");
trainingData.addElement(element);
trainingData.stopRecording();
```
That could ...@fabian Brought up a point about the trainingData class, and its current, stateful API.
It does this:
```cpp
trainingData.startRecording("myLabel");
trainingData.addElement(element);
trainingData.stopRecording();
```
That could cause problems if these functions aren't called in the right order. A safer API would be:
```cpp
trainingData::phrase *myPhrase = trainingData.createPhase(); //returns a pointer to a phrase
myPhrase.addElement(<element>);
```https://gitlab.doc.gold.ac.uk/rapid-mix/RAPID-MIX_API/-/issues/37Design windowed process2017-12-04T16:37:12ZMichael ZbyszyńskiDesign windowed processMZ is about to add a feature to RapidLib to allow users to run "continuous" processing on a window of input frames. In other words, this function gets input data frame-by-frame and buffers it into the right size series and calls dtw.run(...MZ is about to add a feature to RapidLib to allow users to run "continuous" processing on a window of input frames. In other words, this function gets input data frame-by-frame and buffers it into the right size series and calls dtw.run() every n number of frames.
Since this is going to co-exist with similar functions in Rapid-Mix API, we should come up with some general design for this kind of use.
@francisco This is basically what is happening when GVF is in following mode. AFAIK, GVF only does following -- there is no discreet mode for processing a pre-recorded input series.
@Joseph I think that hhmm.filter() is the same type of function as GVF.update() for continuous recognition?
This suggests to me that RapidLib/DTW should handle its own windowing.
I did consider a design that was some kind of windowedML object. That might take a pointer to a specific model and some window parameters. So something like:
```
temporalClassification myModel;
windowedMLObject myWindowThing(*myModel, <window size>, <hopsize>);
myWindowThing.continuousProcessMethod(<new data frame>);
```
That would need better names.
Is there any value in having this kind of object in the RAPID-MIX API?
I'll tag @rebeccaf and @FredB here, too,Michael ZbyszyńskiMichael Zbyszyński