Developer’s Guide

We’re thrilled that you’re interested in contributing to stix2-patterns! Here are some things you should know:

  • contribution-guide.org has great ideas for contributing to any open-source project (not just this one).

  • All contributors must sign a Contributor License Agreement. See CONTRIBUTING.md in the project repository for specifics.

  • If you are planning to implement a major feature (vs. fixing a bug), please discuss with a project maintainer first to ensure you aren’t duplicating the work of someone else, and that the feature is likely to be accepted.

Now, let’s get started!

Setting up a development environment

We recommend using a virtualenv.

1. Clone the repository. If you’re planning to make pull request, you should fork the repository on GitHub and clone your fork instead of the main repo:

git clone https://github.com/yourusername/cti-pattern-validator.git

  1. Install develoment-related dependencies:

cd cti-pattern-validator
pip install -r requirements.txt

  1. Install pre-commit git hooks:

pre-commit install

At this point you should be able to make changes to the code.

Code style

All code should follow PEP 8. We allow for line lengths up to 160 characters, but any lines over 80 characters should be the exception rather than the rule. PEP 8 conformance will be tested automatically by Tox and Travis-CI (see below).

Updating the Grammar

The ANTLR pattern grammar is maintained in the stix2-json-schemas repository. If the grammar changes, the code in this repository should be updated to match. To do so, use the Java ANTLR package to generate new Python source files. (The .jar file is not needed for normal use of the validator).

  1. Download antlr-4.7.1-complete.jar from http://www.antlr.org/download/

  2. Clone the stix2-json-schemas repository or download the STIXPattern.g4 file.

  3. Change to the directory containing the STIXPattern.g4 file.

  4. Run the following command (for STIX v2.1)

    java -jar "/path/to/antlr-4.7.1-complete.jar" -Dlanguage=Python2 STIXPattern.g4 -visitor -o /path/to/cti-pattern-validator/stix2patterns/v21/grammars

  5. Commit the resulting files to git.

Testing

Note

All of the tools mentioned in this section are installed when you run pip install -r requirements.txt.

This project uses pytest for testing. We encourage the use of test-driven development (TDD), where you write (failing) tests that demonstrate a bug or proposed new feature before writing code that fixes the bug or implements the features. Any code contributions should come with new or updated tests.

To run the tests in your current Python environment, use the pytest command from the root project directory:

pytest

This should show all of the tests that ran, along with their status.

You can run a specific test file by passing it on the command line:

pytest stix2patterns/test/v21/test_<xxx>.py

You can also test against the examples provided in the supplied example file. Note that you must specify which version to test.

validate-patterns -v 2.1 -f stix2patterns/test/v21/spec_examples.txt

To ensure that the test you wrote is running, you can deliberately add an assert False statement at the beginning of the test. This is another benefit of TDD, since you should be able to see the test failing (and ensure it’s being run) before making it pass.

tox allows you to test a package across multiple versions of Python. Setting up multiple Python environments is beyond the scope of this guide, but feel free to ask for help setting them up. Tox should be run from the root directory of the project:

tox

We aim for high test coverage, using the coverage.py library. Though it’s not an absolute requirement to maintain 100% coverage, all code contributions must be accompanied by tests. To run coverage and look for untested lines of code, run:

pytest --cov=stix2patterns
coverage html

then look at the resulting report in htmlcov/index.html.

All commits pushed to the master branch or submitted as a pull request are tested with Travis-CI automatically.

Adding a dependency

One of the pre-commit hooks we use in our develoment environment enforces a consistent ordering to imports. If you need to add a new library as a dependency please add it to the known_third_party section of .isort.cfg to make sure the import is sorted correctly.