Developing Streamparse¶
Lein¶
Install Leiningen according to the instructions in the quickstart.
Using Local Clojure Interop Library¶
You can tell lein
to point directly at streamparse’s Clojure repo and use the
code there for all of the interop commands, so that you can test changes while
developing.
To do this, add a directory called checkouts
and symlink it up:
mkdir checkouts
cd checkouts
ln -s ../../../streamparse/jvm streamparse
cd ..
Now, comment out the com.parsely/streamparse
dependency in project.clj
.
It will now pick up the Clojure commands from your local repo. So, now you can
tweak and change them!
Local pip installation¶
In your virtualenv for this project, go into ~/repos/streamparse
(where you
cloned streamparse) and simply run:
python setup.py develop
This will install a streamparse Python version into the virtualenv which is essentially symlinked to your local version.
NOTE: streamparse currently pip installs streamparse’s released version on remote clusters automatically. Therefore, though this will work for local development, you’ll need to push streamparse somewhere pip installable (or change requirements.txt) to make it pick up that version on a remote cluster.
Installing Storm pre-releases¶
You can clone Storm from Github here:
git clone git@github.com:apache/storm.git
There are tags available for releases, e.g.:
git checkout v0.9.2-incubating
To build a local Storm release, use:
mvn install
cd storm-dist/binary
mvn package
These steps will take awhile as they also run Storm’s internal unit and integration tests.
The first line will actually install Storm locally in your maven (.m2) repository. You can confirm this with:
ls ~/.m2/repository/org/apache/storm/storm-core/0.9.2-incubating
You should now be able to change your project.clj
to include a reference to
this new release.
Once you change that, you can run:
lein deps :tree | grep storm
To confirm it is using the upgraded Clojure 1.5.1 (changed in 0.9.2), run:
lein repl