merge readme changes

README.md

@@ -13,7 +13,7 @@
 
 <img src='https://img.shields.io/github/license/beardog108/onionr'> <img src="https://img.shields.io/github/repo-size/beardog108/onionr"> <img src='https://img.shields.io/badge/python%20version%20%F0%9F%90%8D-3.7+-blue'> <img src='https://img.shields.io/github/commit-activity/m/beardog108/onionr'>
 
-<a href='https://www.reddit.com/r/onionr'><img src = 'https://img.shields.io/reddit/subreddit-subscribers/onionr?style=social'></a> <a href='https://twitter.com/onionrnet'><img src='https://img.shields.io/twitter/follow/onionrnet?style=social'></a>
+<a href='https://www.reddit.com/r/onionr'><img src = 'https://img.shields.io/reddit/subreddit-subscribers/onionr?style=social'></a> <a href='https://twitter.com/onionrnet'><img src='https://img.shields.io/twitter/follow/onionrnet?style=social'></a> - [Discord](https://discord.gg/DVF2bEAzrt) - Matrix: #onionr:amorgan.xyz
 
 |                                      |                            |                            |
 | -----------                          | -----------                | -----------                |
@@ -67,13 +67,6 @@ Not yet usable:
 * Instant messaging
 
 
-
-## Watch the talk from BSidesPDX 2019
-
-<a href="https://www.youtube.com/watch?v=mrULtmSkKxg">
-<img src="docs/talk.png" alt="improving anonymous networking talk link" width="600">
-</a>
-
 # Screenshots
 
 <img alt='Node statistics page screenshot' src='docs/onionr-1.png' width=600>
@@ -104,9 +97,10 @@ Master may be unstable, you should use the latest release tag. (checkout via git
 `$ sudo apt install python3-pip python3-dev tor`
 
 * Have python3.7+, python3-pip, Tor (daemon, not browser) installed. python3-dev is recommended.
+* You may need build-essentials or the equivalent of your platform
 * Clone the git repo: `$ git clone https://gitlab.com/beardog/onionr --tags`
 * cd into install direction: `$ cd onionr/`
-* Install the Python dependencies ([virtualenv strongly recommended](https://virtualenv.pypa.io/en/stable/userguide/)): `$ pip3 install --require-hashes -r requirements.txt` (on ARM64 devices like Raspberry Pi's use requirements-ARM.txt instead.)
+* Install the Python dependencies ([virtualenv strongly recommended](https://virtualenv.pypa.io/en/stable/userguide/)): `$ pip3 install --require-hashes -r requirements.txt` (on ARM64 devices like Raspberry Pi 4's use requirements-ARM.txt instead.)
 * (Optional): Install desktop notification dependencies: `$ pip3 install --require-hashes -r requirements-notifications.txt`
 
 (--require-hashes is intended to prevent exploitation via compromise of PyPi/CA certificates)
@@ -124,7 +118,7 @@ Master may be unstable, you should use the latest release tag. (checkout via git
 * Email: beardog [ at ] mailbox.org
 * Twitter: [@onionrnet](https://twitter.com/onionrnet)
 * Onionr Mail: decentralized-fiery-freehearted-skimmer-yodling-topstitch-divorceable-ojibwa-resettlement-infracted-lessor-noninstinctual-leaseholder-counterpoised-couture-skinful
-* Element: #onionr:amorgan.xyz
+* Matrix: #onionr:amorgan.xyz
 * Discord: https://discord.gg/DVF2bEAzrt (Discord is bad for freedom and privacy, this is only provided for convienience)
 
 # Help out
@@ -143,6 +137,12 @@ Everyone is welcome to contribute. Help is wanted for the following:
 * Security review/audit
 * I2P support
 
+## Watch the talk from BSidesPDX 2019
+
+<a href="https://www.youtube.com/watch?v=mrULtmSkKxg">
+<img src="docs/talk.png" alt="improving anonymous networking talk link" width="600">
+</a>
+
 ## Contribute money:
 
 Donating at least $3 gets you cool Onionr stickers. Get in touch if you want them.

docs/dev/running-tests.md

@@ -0,0 +1,44 @@
+# Running Onionr tests
+
+Onionr has four types of tests:
+
+* unittests
+* integration tests
+* selenium tests (web tests)
+* runtime-tests
+
+
+## unittests
+
+Onionr uses Python's built in unittest module. These tests are located in tests/ (top level)
+
+Run all tests with `$ make test`, which will also run integration tests.
+
+Please note that one unittest tests if runtime-tests have passed recently. This is simply a forceful reminder to run those tests as well.
+
+You can also run a single unittest in a loop by using the script scripts/run-unit-test-by-name.py
+
+## integration tests
+
+These tests are pretty basic and test on stdout of Onionr commands.
+
+They are also run from `$ make test`
+
+The runtime-tests do most of the actual integration testing.
+
+## selenium tests
+
+These are browser automation tests to test if the UI is working as how it should for a user.
+
+There's only a couple and they're incomplete, so they can be ignored for now (test manually)
+
+## runtime-tests
+
+These are important. They look into the Onionr client Flask app when Onionr daemon is running and test a bunch of things.
+
+If you do it a lot you should make your own Onionr network (disable official bootstrap)
+
+You run this while the daemon is running (probably should make sure onboarding is done), with `$ onionr.sh runtime-test`
+
+It's necessary to do this before running `$ make test` for unittesting
+

docs/dev/security-mechanisms.md

@@ -0,0 +1,43 @@
+# Onionr Security Mechanisms
+
+
+## bigbrother 👁️
+
+Bigbrother is a cheeky module that uses Python3.8+ sys auditing events to log and/or block certain sensitive events.
+
+It has a little overhead, so one can disable it in config in general.security_auditing
+
+[ChaosWebs.net/blog/preventing-arbitrary-code-execution-in-python38-with-auditing.html](https://chaoswebs.net/blog/preventing-arbitrary-code-execution-in-python38-with-auditing.html)
+
+### Threat model
+
+It is intended to log bugs leaking private file system information, block+log network leaks, and block+log eval-like arbitrary code execution. It is not intended to block malicious browser scripts or malicious Python plugins. It cannot work with subprocesses that do not activate the module.
+
+It's not intended to be bulletproof by any means, but it helps.
+
+### What big brother does
+
+* Disk access checks for disk access outside. Only logs, does not block
+* Network leaks. (Non Tor/LAN) Blocks and logs
+* Arbitrary code execution: logs and blocks non-whitelisted bytecode importing/compiling and subprocesses.
+
+
+## Sybil attacks
+
+As with any decentralized network, sybil nodes could collude to spy or cause mayhem. Due to the gossip nature of Onionr, sybil nodes would have a hard time fully stopping the network. In terms of spying, they could not conclusively prove the origin of messages due to the multiple transport nature of the network and layering behind Tor/etc.
+
+## Tor configuration
+
+When managed by Onionr, Tor has a control port password that gets stored in Onionr config.
+
+Tor is also configured to reject requests to non-onion services, which helps to stop redirect based denial of service attacks.
+
+## Web security
+
+Onionr secures both it's main web APIs with anti-dns-rebinding logic, which validates the host header used in connections to it. This is to prevent exfiltration of data and side channel deanonymization.
+
+Onionr secures the client API with a token that must be passed in most requests, with the exception of static API files. This is to prevent CSRF and side channel deanonymization.
+
+Onionr binds most services to random loopback addresses to reduce all cross-site web attacks, including discovery of Onionr on a computer from a normal website. This is not supported on Mac because Mac does not support non 'typical' loopback addresses.
+
+Onionr has a strict content-security-policy, rejecting all non-localhost requests and denying inline scripts and similar insecure sources.

src/bigbrother/ministry/ofdisk.py

@@ -1,5 +1,23 @@
+"""Onionr - Private P2P Communication.
+
+Log (not block) read/write of non-user data files and non-python lib files
+"""
 from utils.identifyhome import identify_home
 import logger
+"""
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
 
 
 def detect_disk_access(info):