Please follow the quickstart to get Web Security Map on your system.
Additionally the Docker page could be very helpful.
A normal change to the software¶
This is an illustration to show a normal development practices.
We use git, which comes with many pitfalls. Just search your answer on stack exchange first and ask in the group later.
Make sure Web Security Map is up and running, by following the getting started instructions. Then make a branch that describes what you are doing. For example:
git branch documentation_upgrade
Then make any changes to the source code. You’ll see that the devserver automatically restarts after every save.
Once you’re happy with your changes, and you’ve tested it on your environment using a meaningful dataset (eg productiondata), then you can verify (and autofix) the code to our code standards:
Fix any of the remarks it gives, otherwise your changes will not be added to the master branch.
You can only commit files to the master branch if you’re up to date with it’s code:
git pull --rebase origin master
Follow the instructions on screen to fix merge conflicts if any.
When you are happy, review your changes and remove any temporary files using the instructions given:
If you like the files that are changed, add all changes to be merged:
git add -A
Then commit them:
git commit -m "a short description why you changed something"
Then push them to the server:
The push command will give you a link to file a merge request. Meanwhile the build servers are checking your code before merging.
Follow the merge request link to create the actual merge request. Share it on the chat.websecuritymap.org channel for review and feedback.
Once the feedback is processed (if needed at all) you can merge the code. If you can’t, other members of the project can. So ask.
The web security map command won’t start and i get some weird errors¶
Make sure you’ve got an up to date development environment. You can do so by running the following commands:
Rebuild the environment:
Get all requirements and development requirements:
Code quality / Testing¶
This project sticks to default pycodestyle/pyflakes configuration to maintain code quality.
To run code quality checks and unit tests run:
For a comprehensive test run:
make check test test_datasets
To make life easier you can use
isort before running
make to automatically fix most style issues:
To run only a specific test use:
make test testargs="-k test_name"
To only run a specific test suite use for example:
make test testargs=tests/scanners/test_dummy.py
A coverage report is generated after running tests, on OSX it can be viewed using:
Pytest allows to drop into Python debugger when a tests fails. To enable run:
Besides quality checks and unit tests there are also integration and system testing frameworks available.
These frameworks will run in the CI system but not by default when running
make due to their dependencies.
To run these testsuites make sure Docker is installed and running and run either:
Direnv / Virtualenv¶
This project has direnv configuration to automatically manage the Python
virtual environment. Install direnv and run
direnv allow to enable it initially. After this the environment will by automatically loaded/unloaded every time you enter/leave the project directory.
If you don’t want to use Direnv be sure to source the
.envrc file manually every time you want to work on the project:
Python dependencies managment¶
Dependencies and settings for this project are managed using Pip-tools which is simple tool build on top of Pip and
Dependencies are added to (or removed from) the
requirements.in file (
requirements-dev.in for development dependencies). The
pip-compile command will then be used to parse this file and generate a list of all (recursive)dependencies to install with exact versions pinnen and put this in the respective
Any update to a
.in file will automatically trigger a Make target to update
.txt files and install new dependencies when running Make commands.
Version pinning in
.in files should be kept to a minimum and if it is required should be accompanied by a rationale/deadline comment.
pip-compile check for new versions of all dependencies within the version boundaries of
.in files run:
.in and the
.txt files should be commited to Git. The
.txt files are used during installation to ensure dependencies are predictable accross environments.