ai4cr
Artificial Intelligence for Crystal (based on https://github.com/SergioFierens/ai4r)
Installation
Add this to your application's shard.yml
:
dependencies:
ai4cr:
github: drhuffman12/ai4cr
branch: master
Usage
require "ai4cr"
So far, only Ai4cr::NeuralNetwork::Backpropagation and related tests have been ported.
NOTE marshal_dump
and marshal_load
from ai4r have been replaced by to_json
and from_json
instead, e.g.:
# Create and save a net
net = Ai4cr::NeuralNetwork::Backpropagation.new(...)
File.write("../ai4cr_ui/db/seeds/BackpropagationNet.new.json",net.to_json)
# Train and save a net
net.train(some_input, expected_output)
File.write("../ai4cr_ui/db/seeds/BackpropagationNet.trained.json",net.to_json)
# Verify serialization in a spec
json = net.to_json
net2 = Ai4cr::NeuralNetwork::Backpropagation.from_json(json)
assert_approximate_equality_of_nested_list net.weights, net2.weights, 0.000000001
Optimizations
- Compiler
Use --release
for more code optimizations during the compilation steps.
e.g.: time crystal spec --release
- Multithreading
Use the -Dpreview_mt
(for crystal build
or -D preview_mt
for crystal spec
) flag for multithreading.
e.g.:
# build:
time CRYSTAL_WORKERS=14 crystal build examples/rnn_simple_manager_example_relu.cr --release -D preview_mt
# run (and log to tmp folder):
time CRYSTAL_WORKERS=24 ./rnn_simple_manager_example_relu > tmp/log_relu.txt 2>&1
(Personally, as for how many CRYSTAL_WORKERS
, I'd recommend keep it to less than the number of cores in your CPU, so that you leave at least one or two cores for the OS and apps.)
See also:
- https://crystal-lang.org/2019/09/23/crystal-0.31.0-released.html
- https://crystal-lang.org/2019/09/06/parallelism-in-crystal.html
Comparison benchmarks
Clear Crystal cache
- Shards:
rm -rf ~/.cache/shards/
- Compiler:
rm -rf ~/.cache/crystal/
Run benchmarks
REMINDER: Running Crystal in a Docker container (at least used to) runs slower than running Crystal on bare-metal. So, for more performance, run it outside of a Docker container.
To build and run them and see the debugging output (DEBUG=1):
DEBUG=1 crystal spec spec_bench
Example output:
To dig deeper into performance refinement:
crystal build --release src/mini_nets_vs_backprop
mkdir -p tmp/
valgrind --tool=callgrind --cache-sim=yes --branch-sim=yes --callgrind-out-file=tmp/mini_nets_vs_backprop.out ./mini_nets_vs_backprop
Importing and Exporting as JSON data
NOTE Parsing to/from JSON can cause slight discrepancies in the float values. This difference shouldn't matter much, but in case you're interested:
Parsing Float64 values to/from JSON is sometimes slightly off, but only by about 0.0000000000000001%. This seems more likely when scientific notation gets involved, which could happen during the first training session, but becomes more likely the more times the net is trained.
For example, see below example code and note discrepancies after the "033410668390063" parts:
require "json"
# a = 6.033410668390063e-5
# a = 6.0334106683900631e-5
# a = 0.60334106683900634
a = 0.60334106683900631
b = a.to_json
c = JSON.parse(b)
puts "a,b,c == #{[a, b, c]}"
# GIVEN "a = 6.033410668390063e-5", puts => a == 6.033410668390063e-5; b == "6.033410668390063e-5"; c == 6.0334106683900634e-5 (off)
# GIVEN "a = 6.0334106683900631e-5", puts => a == 6.033410668390063e-5 (off); b == "6.033410668390063e-5" (off); c == 6.0334106683900634e-5 (off)
# GIVEN "a = 0.60334106683900634", puts => a == 0.6033410668390063 (off); b == "0.6033410668390063" (off); c == 0.6033410668390063 (off)
# GIVEN "a = 0.60334106683900631", puts => a == 0.6033410668390063 (off); b == "0.6033410668390063" (off); c == 0.6033410668390063 (off)
Roadmap
- [x] Generate an error history plot using
AsciiBarCharter
andErrorStats#history
, e.g.:
plot: '▇▊▂_▅▅▅_▅_▅▅▅▅_▅▅__▅_▅____▅___'
(Run crystal spec spec_examples
to see more error history plot examples. NOTE: These run short training sessions, so some tests are likely to fail some of the time.)
-
[ ] Add Cmn ("Connectable Mini Networks") (WIP)
- [x] simple benchmark comparisons
- [x] Learning Styles
- [x] Pelu
- [x] Relu
- [x] Sigmoid
- [x] Tanh
- [x] MiniNet
- [x] MiniNetConcerns modules
- [x] JSON importable/exportable
- [x] can use various 'Learning Styles'
- [ ] (?) move 'Learning Styles'-specific methods from MiniNet into Enum
- [ ] misc Connectable Net Sets (WIP)
- [x] Chain
- [ ] RNN
- [x] RnnSimple
- [ ] Code to split and train on a 'sequence of input and output data'
- [ ] (TBD)
- [ ] ...
- [ ] (TBD)
- [ ] ...
-
[ ] Port from
ai4r
:- [ ] classifiers
- [ ] classifier.rb
- [ ] hyperpipes.rb
- [ ] ib1.rb
- [ ] id3.rb
- [ ] multilayer_perceptron.rb
- [ ] naive_bayes.rb
- [ ] one_r.rb
- [ ] prism.rb
- [ ] simple_linear_regression.rb
- [ ] votes.rb
- [ ] zero_r.rb
- [ ] clusterers
- [ ] average_linkage.rb
- [ ] bisecting_k_means.rb
- [ ] centroid_linkage.rb
- [ ] clusterer.rb
- [ ] complete_linkage.rb
- [ ] diana.rb
- [ ] k_means.rb
- [ ] median_linkage.rb
- [ ] single_linkage.rb
- [ ] ward_linkage_hierarchical.rb
- [ ] ward_linkage.rb
- [ ] weighted_average_linkage.rb
- [ ] data
- [ ] data_set.rb
- [ ] parameterizable.rb
- [ ] proximity.rb
- [ ] statistics.rb
- [ ] experiment
- [ ] classifier_evaluator.rb
- [ ] genetic_algorithm
- [ ] genetic_algorithm.rb
- [ ] neural_network
- [x] backpropagation.rb
- [ ] hopfield.rb
- [ ] som
- [ ] layer.rb
- [ ] node.rb
- [ ] som.rb
- [ ] two_phase_layer.rb
- [ ] classifiers
If you'd like another class of Ai4r ported, feel free to submit a new issue.
Contributing
- Fork it ( https://github.com/drhuffman12/ai4cr/fork )
- Create your feature branch (git checkout -b my-new-feature)
- Commit your changes (git commit -am 'Add some feature')
- Push to the branch (git push origin my-new-feature)
- Create a new Pull Request
(Re-)Format
docker-compose run app scripts/reformat
Build
# for a cleaner build:
docker-compose build --force-rm --no-cache --pull
# normally:
docker-compose build
Show version
docker-compose run app scripts/version_info
Add Ameba bin file
TODO Find out why bin/ameba
runs fine in CI but not locally.
docker-compose run app shards update
Test
docker-compose run app crystal spec
Docker console
docker-compose run app sh
ICR console
NOTE This isn't yet Crystal 1.0 compatible!
icr -r ./src/ai4cr
See also
-
The docs at: https://drhuffman12.github.io/ai4cr/
-
The specs and https://github.com/SergioFierens/ai4r for more info.
Contributors
- drhuffman12 Daniel Huffman - creator, maintainer
Example Spec Results (running on a Lenovo Ideapad y700 w/ i7-6700HQ)
I included the triangle-square-cross training example as test cases.
As expected, the net sometimes correctly guesses all of the examples, more or less, depending on how many times it is trained and various other (random) factors.
Below is an example of the net successfully recognizing all nine test cases.
Setup
$ cd $MY_DEV_FOLDER
$ git clone https://github.com/drhuffman12/ai4cr.git
$ cd ai4cr
$ docker-compose build
These should NEVER fail!
For any tests that should (well, usually) NEVER fail (e.g.: in spite of sufficient training), put them into spec
, and run them via:
$ docker-compose run app scripts/test_always
..............................
Finished in 4.01 milliseconds
30 examples, 0 failures, 0 errors, 0 pending
Execute: 00:00:00.010855717
These will probably SOMETIMES fail!
For any tests that could fails sometimes (e.g.: if not trained enough), put them into spec_examples
, and run them via:
$ docker-compose run app scripts/test_sometimes
.............
Finished in 6.76 seconds
16 examples, 0 failures, 0 errors, 0 pending
Execute: 00:00:06.769663359
NOTE That time, it took less than a second to build. I did notice that it took about 10 seconds to build the first run and only less than a second each successive run.
More debug info
TODO Convert this to using the logger.
To see more debug info (all those extra puts commands) during testing, Set the DEBUG
env variable to 1
, e.g.
DEBUG=1 crystal spec