Merge branch 'crypto'

.gitignore

@@ -6,3 +6,6 @@ onionr/*.pyc
 onionr/*.log
 onionr/data/hs/hostname
 onionr/data/*
+onionr/data-backup/*
+onionr/gnupg/*
+run.sh

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "onionr/bitpeer"]
+	path = onionr/bitpeer
+	url = https://github.com/beardog108/bitpeer.py

.travis.yml

@@ -5,4 +5,4 @@ python:
 install:
         - sudo apt install gnupg tor
         - pip install -r requirements.txt
-script: ./test.sh
+script: make test

Makefile

@@ -0,0 +1,29 @@
+.DEFAULT_GOAL := setup
+
+setup:
+	sudo pip3 install -r requirements.txt
+
+install:
+	sudo rm -rf /usr/share/onionr/
+	sudo rm -f /usr/bin/onionr
+	sudo cp -rp ./onionr /usr/share/onionr
+	sudo sh -c "echo \"#!/bin/sh\ncd /usr/share/onionr/\n./onionr.py \\\"\\\$$@\\\"\" > /usr/bin/onionr"
+	sudo chmod +x /usr/bin/onionr
+	sudo chown -R `whoami` /usr/share/onionr/
+
+uninstall:
+	sudo rm -rf /usr/share/onionr
+	sudo rm -f /usr/bin/onionr
+
+test:
+	@rm -rf onionr/data-backup
+	@mv onionr/data onionr/data-backup | true > /dev/null 2>&1
+	-@cd onionr; ./tests.py
+	@rm -rf onionr/data
+	@mv onionr/data-backup onionr/data | true > /dev/null 2>&1
+
+reset:
+	rm -f onionr/data/blocks/*.dat | true > /dev/null 2>&1
+	rm -f onionr/data/peers.db | true > /dev/null 2>&1
+	rm -f onionr/data/blocks.db | true > /dev/null 2>&1
+	rm -f onionr/data/address.db | true > /dev/null 2>&1

docs/api.md

@@ -1,3 +1,9 @@
+BLOCK HEADERS (simple ID system to identify block type)
+-----------------------------------------------
+-crypt- (encrypted block)
+-bin- (binary file)
+-txt- (plaintext)
+
 HTTP API
 ------------------------------------------------
 /client/ (Private info, not publicly accessible)

docs/onionr-draft.md

@@ -1,72 +1,51 @@
-# Onionr Protocol Spec
+# Onionr Protocol Spec v2
 
-A social network/microblogging platform for Tor & I2P
+A P2P platform for Tor & I2P
-
-Draft Dec 25 2017
 
 # Overview
 
 Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
 There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
-User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
+User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
-Clients consolidate feeds from peers into 1 “timeline” using RSS format.
+Private blocks are only able to be read by the intended peer.
-Private messages are only accessible by the intended peer based on the PGP id.
+All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
-Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
+
-All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
 ## Goals:
-    • Selective sharing of information with friends & public
+    • Selective sharing of information
     • Secure & semi-anonymous direct messaging
     • Forward secrecy
     • Defense in depth
-    • Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
+    • Data should be secure for years to come
     • Decentralization
     * Avoid browser-based exploits that plague similar software
     * Avoid timing attacks & unexpected metadata leaks
-## Assumptions:
+
-    • Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
-    • All traffic is logged indefinitely by powerful adversaries
 ## Protocol
-Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
+
+Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
+
+Blocks may be encrypted using Curve25519.
+
 ## Connections
-    When a node first comes online, it attempts to bootstrap using a default list provided by a client.
-    When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
-    HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
-    All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
-    Peer Types
-        * Friends:
-            * Encrypted ‘friends only’ posts to one another
-            * Usually less strict rate & storage limits
-            * OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
-        • Strangers:
-            * Used for storage of encrypted or public information
-            * Can only read public posts
-            * Usually stricter rate & storage limits
-## Data Storage/Delivery
 
-    Posts (public or friends only) are stored across the network.
+When a node first comes online, it attempts to bootstrap using a default list provided by a client.
-    Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
+When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
-    Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
-    Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
-    Posts
-    Posts can contain text and images. All posts MUST be time stamped.
-    Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
-    All received posts must be verified to be stored and/or displayed to the user.
 
-    All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
+Salsa20 keys are regenerated either every X many communications with a peer or every X minutes. 
 
-     Posts have two settings:
+Every 100kb or every 2 hours is a recommended default.
-        • Friends only:
-            ◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
-        • Public:
-            ◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
-## Private Messages
 
-    Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
+All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
-    When both peers are online messages SHOULD be dispatched directly between peers.
+Peer Types
-    All messages must be verified prior to being displayed.
+    * Friends:
+        * Encrypted ‘friends only’ posts to one another
+        * Usually less strict rate & storage limits
+    * Strangers:
+        * Used for storage of encrypted or public information
+        * Can only read public posts
+        * Usually stricter rate & storage limits
 
-    Clients SHOULD allow configurable message padding.
 ## Spam mitigation
 
 To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
-Rate limits can be strict, as Onionr is not intended to be an instant messaging application.
+Rate limits can be strict, as Onionr is not intended to be an instant messaging application.

onionr/api.py

@@ -20,45 +20,51 @@
 import flask
 from flask import request, Response, abort
 from multiprocessing import Process
-import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
+import sys, random, threading, hmac, hashlib, base64, time, math, os, logger, config
 
 from core import Core
-import onionrutils
+import onionrutils, onionrcrypto
 class API:
-    ''' Main http api (flask)'''
+    '''
+        Main HTTP API (Flask)
+    '''
     def validateToken(self, token):
         '''
-        Validate if the client token (hmac) matches the given token
+            Validate that the client token (hmac) matches the given token
         '''
         if self.clientToken != token:
             return False
         else:
             return True
 
-    def __init__(self, config, debug):
+    def __init__(self, debug):
-        ''' Initialize the api server, preping variables for later use
-        This initilization defines all of the API entry points and handlers for the endpoints and errors
-
-        This also saves the used host (random localhost IP address) to the data folder in host.txt
         '''
-        if os.path.exists('dev-enabled'):
+            Initialize the api server, preping variables for later use
+
+            This initilization defines all of the API entry points and handlers for the endpoints and errors
+            This also saves the used host (random localhost IP address) to the data folder in host.txt
+        '''
+
+        config.reload()
+
+        if config.get('devmode', True):
             self._developmentMode = True
             logger.set_level(logger.LEVEL_DEBUG)
-            logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
         else:
             self._developmentMode = False
             logger.set_level(logger.LEVEL_INFO)
 
-        self.config = config
         self.debug = debug
         self._privateDelayTime = 3
         self._core = Core()
+        self._crypto = onionrcrypto.OnionrCrypto(self._core)
         self._utils = onionrutils.OnionrUtils(self._core)
         app = flask.Flask(__name__)
-        bindPort = int(self.config['CLIENT']['PORT'])
+        bindPort = int(config.get('client')['port'])
         self.bindPort = bindPort
-        self.clientToken = self.config['CLIENT']['CLIENT HMAC']
+        self.clientToken = config.get('client')['client_hmac']
-        logger.debug('Your HMAC token: ' + logger.colors.underline + self.clientToken)
+        if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
+            logger.debug('Your HMAC token: ' + logger.colors.underline + self.clientToken)
 
         if not debug and not self._developmentMode:
             hostNums = [random.randint(1, 255), random.randint(1, 255), random.randint(1, 255)]
@@ -72,9 +78,10 @@ class API:
         @app.before_request
         def beforeReq():
             '''
-            Simply define the request as not having yet failed, before every request.
+                Simply define the request as not having yet failed, before every request.
             '''
             self.requestFailed = False
+
             return
 
         @app.after_request
@@ -87,6 +94,7 @@ class API:
             resp.headers["Content-Security-Policy"] = "default-src 'none'"
             resp.headers['X-Frame-Options'] = 'deny'
             resp.headers['X-Content-Type-Options'] = "nosniff"
+
             return resp
 
         @app.route('/client/')
@@ -112,6 +120,7 @@ class API:
             elapsed = endTime - startTime
             if elapsed < self._privateDelayTime:
                 time.sleep(self._privateDelayTime - elapsed)
+
             return resp
 
         @app.route('/public/')
@@ -125,14 +134,14 @@ class API:
                 pass
             elif action == 'ping':
                 resp = Response("pong!")
-            elif action == 'setHMAC':
+            elif action == 'getHMAC':
-                pass
+                resp = Response(self._crypto.generateSymmetric())
+            elif action == 'getSymmetric':
+                resp = Response(self._crypto.generateSymmetric())
             elif action == 'getDBHash':
                 resp = Response(self._utils.getBlockDBHash())
             elif action == 'getBlockHashes':
                 resp = Response(self._core.getBlockList())
-            elif action == 'getPGP':
-                resp = Response(self._utils.exportMyPubkey())
             # setData should be something the communicator initiates, not this api
             elif action == 'getData':
                 resp = self._core.getData(data)
@@ -140,6 +149,16 @@ class API:
                     abort(404)
                     resp = ""
                 resp = Response(resp)
+            elif action == 'pex':
+                response = ','.join(self._core.listAdders())
+                if len(response) == 0:
+                    response = 'none'
+                resp = Response(response)
+            elif action == 'kex':
+                response = ','.join(self._core.listPeers())
+                if len(response) == 0:
+                    response = 'none'
+                resp = Response(response)
             else:
                 resp = Response("")
 
@@ -149,26 +168,36 @@ class API:
         def notfound(err):
             self.requestFailed = True
             resp = Response("")
-            #resp.headers = getHeaders(resp)
+
             return resp
+
         @app.errorhandler(403)
         def authFail(err):
             self.requestFailed = True
             resp = Response("403")
+
             return resp
+
         @app.errorhandler(401)
         def clientError(err):
             self.requestFailed = True
             resp = Response("Invalid request")
+
             return resp
+        if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
+            logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
 
-        logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
+        try:
-        logger.debug('Client token: ' + logger.colors.underline + self.clientToken)
+            app.run(host=self.host, port=bindPort, debug=True, threaded=True)
-
+        except Exception as e:
-        app.run(host=self.host, port=bindPort, debug=True, threaded=True)
+            logger.error(str(e))
+            logger.fatal('Failed to start client on ' + self.host + ':' + str(bindPort) + ', exiting...')
+            exit(1)
 
     def validateHost(self, hostType):
-        ''' Validate various features of the request including:
+        '''
+            Validate various features of the request including:
+
             If private (/client/), is the host header local?
             If public (/public/), is the host header onion or i2p?
 

onionr/bitpeer

@@ -0,0 +1 @@
+Subproject commit a74e826e9c69e643ead7950f9f76a05ab8664ddc

onionr/btc.py

@@ -20,8 +20,9 @@
 from bitpeer.node import *
 from bitpeer.storage.shelve import ShelveStorage
 import logging, time
+import socks, sys
 class OnionrBTC:
-	def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613):
+	def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613, torP=9050):
 		stream = logging.StreamHandler()
 		logger = logging.getLogger('halfnode')
 		logger.addHandler(stream)
@@ -29,9 +30,15 @@ class OnionrBTC:
 
 		LASTBLOCK = lastBlock
 		LASTBLOCKINDEX = lastHeight
-		self.node = Node ('BTC', ShelveStorage ('./btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX)
+		self.node = Node ('BTC', ShelveStorage ('data/btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX, torPort=torP)
 
 		self.node.bootstrap ()
 		self.node.connect ()
 		self.node.loop ()
 
+if __name__ == "__main__":
+    torPort = int(sys.argv[1])
+    bitcoin = OnionrBTC(torPort)
+    while True:
+        print(bitcoin.node.getBlockHash(bitcoin.node.getLastBlockHeight())) # Using print on purpose, do not change to logger
+        time.sleep(5)

onionr/colors.py

@@ -1,23 +0,0 @@
-'''
-Simply define terminal control codes (mainly colors)
-'''
-class Colors:
-    def __init__(self):
-        '''
-        PURPLE='\033[95m'
-        BLUE='\033[94m'
-        GREEN='\033[92m'
-        YELLOW='\033[93m'
-        RED='\033[91m'
-        BOLD='\033[1m'
-        UNDERLINE='\033[4m'
-        RESET="\x1B[m"
-        '''
-        self.PURPLE='\033[95m'
-        self.BLUE='\033[94m'
-        self.GREEN='\033[92m'
-        self.YELLOW='\033[93m'
-        self.RED='\033[91m'
-        self.BOLD='\033[1m'
-        self.UNDERLINE='\033[4m'
-        self.RESET="\x1B[m"

onionr/communicator.py

@@ -19,93 +19,144 @@ and code to operate as a daemon, getting commands from the command queue databas
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sqlite3, requests, hmac, hashlib, time, sys, os, logger
+import sqlite3, requests, hmac, hashlib, time, sys, os, math, logger, urllib.parse, random
-import core, onionrutils
+import core, onionrutils, onionrcrypto, onionrproofs, btc, config, onionrplugins as plugins
+
 class OnionrCommunicate:
     def __init__(self, debug, developmentMode):
-        ''' OnionrCommunicate
-
-        This class handles communication with nodes in the Onionr network.
         '''
+            OnionrCommunicate
+
+            This class handles communication with nodes in the Onionr network.
+        '''
+
         self._core = core.Core()
         self._utils = onionrutils.OnionrUtils(self._core)
+        self._crypto = onionrcrypto.OnionrCrypto(self._core)
+        '''
+        logger.info('Starting Bitcoin Node... with Tor socks port:' + str(sys.argv[2]))
+        try:
+            self.bitcoin = btc.OnionrBTC(torP=int(sys.argv[2]))
+        except _gdbm.error:
+            pass
+        logger.info('Bitcoin Node started, on block: ' + self.bitcoin.node.getBlockHash(self.bitcoin.node.getLastBlockHeight()))
+        '''
+        #except:
+        #logger.fatal('Failed to start Bitcoin Node, exiting...')
+        #exit(1)
+
         blockProcessTimer = 0
         blockProcessAmount = 5
         heartBeatTimer = 0
-        heartBeatRate = 10
+        heartBeatRate = 5
+        pexTimer = 5 # How often we should check for new peers
+        pexCount = 0
         logger.debug('Communicator debugging enabled.')
         torID = open('data/hs/hostname').read()
 
-        # get our own PGP fingerprint
+        self.peerData = {} # Session data for peers (recent reachability, speed, etc)
-        fingerprintFile = 'data/own-fingerprint.txt'
+
-        if not os.path.exists(fingerprintFile):
-            self._core.generateMainPGP(torID)
-        with open(fingerprintFile,'r') as f:
-            self.pgpOwnFingerprint = f.read()
-        logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
         if os.path.exists(self._core.queueDB):
             self._core.clearDaemonQueue()
+
+        # Loads in and starts the enabled plugins
+        plugins.reload()
+
         while True:
             command = self._core.daemonQueue()
             # Process blocks based on a timer
             blockProcessTimer += 1
             heartBeatTimer += 1
+            pexCount += 1
+            if pexTimer == pexCount:
+                self.getNewPeers()
+                pexCount = 0
             if heartBeatRate == heartBeatTimer:
                 logger.debug('Communicator heartbeat')
                 heartBeatTimer = 0
             if blockProcessTimer == blockProcessAmount:
                 self.lookupBlocks()
-                self._core.processBlocks()
+                self.processBlocks()
                 blockProcessTimer = 0
-            #logger.debug('Communicator daemon heartbeat')
             if command != False:
                 if command[0] == 'shutdown':
-                    logger.warn('Daemon recieved exit command.')
+                    logger.info('Daemon recieved exit command.')
                     break
             time.sleep(1)
-        return
-    def getRemotePeerKey(self, peerID):
-        '''This function contacts a peer and gets their main PGP key.
 
-        This is safe because Tor or I2P is used, but it does not ensure that the person is who they say they are
-        '''
-        url = 'http://' + peerID + '/public/?action=getPGP'
-        r = requests.get(url, headers=headers)
-        response = r.text
-        return response
-    def shareHMAC(self, peerID, key):
-        '''This function shares an HMAC key to a peer
-        '''
         return
-    def getPeerProof(self, peerID):
+
-        '''This function gets the current peer proof requirement'''
+    def getNewPeers(self):
-        return
+        '''
-    def sendPeerProof(self, peerID, data):
+            Get new peers
-        '''This function sends the proof result to a peer previously fetched with getPeerProof'''
+        '''
+        peersCheck = 5 # Amount of peers to ask for new peers + keys
+        peersChecked = 0
+        peerList = list(self._core.listAdders()) # random ordered list of peers
+        newKeys = []
+        newAdders = []
+        if len(peerList) > 0:
+            maxN = len(peerList) - 1
+        else:
+            peersCheck = 0
+            maxN = 0
+
+        if len(peerList) > peersCheck:
+            peersCheck = len(peerList)
+
+        while peersCheck > peersChecked:
+            i = random.randint(0, maxN)
+            logger.info('Using ' + peerList[i] + ' to find new peers')
+            try:
+                newAdders = self.performGet('pex', peerList[i], skipHighFailureAddress=True)
+                self._utils.mergeAdders(newAdders)
+            except requests.exceptions.ConnectionError:
+                logger.info(peerList[i] + ' connection failed')
+                continue
+            else:
+                try:
+                    logger.info('Using ' + peerList[i] + ' to find new keys')
+                    newKeys = self.performGet('kex', peerList[i], skipHighFailureAddress=True)
+                    # TODO: Require keys to come with POW token (very large amount of POW)
+                    self._utils.mergeKeys(newKeys)
+                except requests.exceptions.ConnectionError:
+                    logger.info(peerList[i] + ' connection failed')
+                    continue
+                else:
+                    peersChecked += 1
         return
 
     def lookupBlocks(self):
-        '''Lookup blocks and merge new ones'''
+        '''
-        peerList = self._core.listPeers()
+            Lookup blocks and merge new ones
+        '''
+        peerList = self._core.listAdders()
         blocks = ''
         for i in peerList:
-            lastDB = self._core.getPeerInfo(i, 'blockDBHash')
+            lastDB = self._core.getAddressInfo(i, 'DBHash')
             if lastDB == None:
                 logger.debug('Fetching hash from ' + i + ' No previous known.')
             else:
-                logger.debug('Fetching hash from ' + i + ', ' + lastDB + ' last known')
+                logger.debug('Fetching hash from ' + str(i) + ', ' + lastDB + ' last known')
             currentDB = self.performGet('getDBHash', i)
+            if currentDB != False:
+                logger.debug(i + " hash db (from request): " + currentDB)
+            else:
+                logger.warn("Error getting hash db status for " + i)
             if currentDB != False:
                 if lastDB != currentDB:
                     logger.debug('Fetching hash from ' + i + ' - ' + currentDB + ' current hash.')
                     blocks += self.performGet('getBlockHashes', i)
-            if currentDB != lastDB:
                 if self._utils.validateHash(currentDB):
-                    self._core.setPeerInfo(i, "blockDBHash", currentDB)
+                    self._core.setAddressInfo(i, "DBHash", currentDB)
-                else:
+        if len(blocks.strip()) != 0:
-                    logger.warn("Peer " + i + " returned malformed hash")
+            logger.debug('BLOCKS:' + blocks)
         blockList = blocks.split('\n')
         for i in blockList:
+            if len(i.strip()) == 0:
+                continue
+            if self._utils.hasBlock(i):
+                continue
             logger.debug('Exchanged block (blockList): ' + i)
             if not self._utils.validateHash(i):
                 # skip hash if it isn't valid
@@ -114,31 +165,100 @@ class OnionrCommunicate:
             else:
                 logger.debug('Adding ' +  i + ' to hash database...')
                 self._core.addToBlockDB(i)
+
         return
 
-    def performGet(self, action, peer, data=None, type='tor'):
+    def processBlocks(self):
-        '''Performs a request to a peer through Tor or i2p (currently only tor)'''
+        '''
+            Work with the block database and download any missing blocks
+
+            This is meant to be called from the communicator daemon on its timer.
+        '''
+
+        for i in self._core.getBlockList(True).split("\n"):
+            if i != "":
+                logger.warn('UNSAVED BLOCK: ' + i)
+                data = self.downloadBlock(i)
+
+        return
+
+    def downloadBlock(self, hash):
+        '''
+            Download a block from random order of peers
+        '''
+
+        peerList = self._core.listAdders()
+        blocks = ''
+        for i in peerList:
+            hasher = hashlib.sha3_256()
+            data = self.performGet('getData', i, hash)
+            if data == False or len(data) > 10000000:
+                continue
+            hasher.update(data.encode())
+            digest = hasher.hexdigest()
+            if type(digest) is bytes:
+                digest = digest.decode()
+            if digest == hash.strip():
+                self._core.setData(data)
+                if data.startswith('-txt-'):
+                    self._core.setBlockType(hash, 'txt')
+                logger.info('Successfully obtained data for ' + hash)
+                if len(data) < 120:
+                    logger.debug('Block text:\n' + data)
+            else:
+                logger.warn("Failed to validate " + hash)
+
+        return
+
+    def urlencode(self, data):
+        '''
+            URL encodes the data
+        '''
+
+        return urllib.parse.quote_plus(data)
+
+    def performGet(self, action, peer, data=None, skipHighFailureAddress=False, peerType='tor'):
+        '''
+            Performs a request to a peer through Tor or i2p (currently only Tor)
+        '''
+
         if not peer.endswith('.onion') and not peer.endswith('.onion/'):
             raise PeerError('Currently only Tor .onion peers are supported. You must manually specify .onion')
+
+        # Store peer in peerData dictionary (non permanent)
+        if not peer in self.peerData:
+            self.peerData[peer] = {'connectCount': 0, 'failCount': 0, 'lastConnectTime': math.floor(time.time())}
         socksPort = sys.argv[2]
         '''We use socks5h to use tor as DNS'''
-        proxies = {'http': 'socks5h://127.0.0.1:' + str(socksPort), 'https': 'socks5h://127.0.0.1:' + str(socksPort)}
+        proxies = {'http': 'socks5://127.0.0.1:' + str(socksPort), 'https': 'socks5://127.0.0.1:' + str(socksPort)}
         headers = {'user-agent': 'PyOnionr'}
-        url = 'http://' + peer + '/public/?action=' + action
+        url = 'http://' + peer + '/public/?action=' + self.urlencode(action)
         if data != None:
-            url = url + '&data=' + data
+            url = url + '&data=' + self.urlencode(data)
         try:
-            r = requests.get(url, headers=headers, proxies=proxies)
+            if skipHighFailureAddress and self.peerData[peer]['failCount'] > 10:
+                retData = False
+                logger.debug('Skipping ' + peer + ' because of high failure rate')
+            else:
+                logger.debug('Contacting ' + peer + ' on port ' + socksPort)
+                r = requests.get(url, headers=headers, proxies=proxies, timeout=(15, 30))
+                retData = r.text
         except requests.exceptions.RequestException as e:
             logger.warn(action + " failed with peer " + peer + ": " + str(e))
-            return False
+            retData = False
-        return r.text
+
+        if not retData:
+            self.peerData[peer]['failCount'] += 1
+        else:
+            self.peerData[peer]['connectCount'] += 1
+            self.peerData[peer]['lastConnectTime'] = math.floor(time.time())
+        return retData
 
 
 shouldRun = False
 debug = True
 developmentMode = False
-if os.path.exists('dev-enabled'):
+if config.get('devmode', True):
     developmentMode = True
 try:
     if sys.argv[1] == 'run':
@@ -149,4 +269,5 @@ if shouldRun:
     try:
         OnionrCommunicate(debug, developmentMode)
     except KeyboardInterrupt:
+        sys.exit(1)
         pass

onionr/config.py

@@ -0,0 +1,114 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with configuration management.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import os, json, logger
+
+_configfile = os.path.abspath('data/config.json')
+_config = {}
+
+def get(key, default = None):
+    '''
+        Gets the key from configuration, or returns `default`
+    '''
+
+    if is_set(key):
+        return get_config()[key]
+    return default
+
+def set(key, value = None, savefile = False):
+    '''
+        Sets the key in configuration to `value`
+    '''
+
+    global _config
+    if value is None:
+        del _config[key]
+    else:
+        _config[key] = value
+
+    if savefile:
+        save()
+
+def is_set(key):
+    return key in get_config() and not get_config()[key] is None
+
+def check():
+    '''
+        Checks if the configuration file exists, creates it if not
+    '''
+
+    try:
+        if not os.path.exists(os.path.dirname(get_config_file())):
+            os.path.mkdirs(os.path.dirname(get_config_file()))
+        if not os.path.isfile(get_config_file()):
+            open(get_config_file(), 'a', encoding="utf8").close()
+            save()
+    except:
+        logger.warn('Failed to check configuration file.')
+
+def save():
+    '''
+        Saves the configuration data to the configuration file
+    '''
+
+    check()
+    try:
+        with open(get_config_file(), 'w', encoding="utf8") as configfile:
+            json.dump(get_config(), configfile, indent=2, sort_keys=True)
+    except:
+        logger.warn('Failed to write to configuration file.')
+
+def reload():
+    '''
+        Reloads the configuration data in memory from the file
+    '''
+
+    check()
+    try:
+        with open(get_config_file(), 'r', encoding="utf8") as configfile:
+            set_config(json.loads(configfile.read()))
+    except:
+        logger.warn('Failed to parse configuration file.')
+
+def get_config():
+    '''
+        Gets the entire configuration as an array
+    '''
+    return _config
+
+def set_config(config):
+    '''
+        Sets the configuration to the array in arguments
+    '''
+    global _config
+    _config = config
+
+def get_config_file():
+    '''
+        Returns the absolute path to the configuration file
+    '''
+    return _configfile
+
+def set_config_file(configfile):
+    '''
+        Sets the path to the configuration file
+    '''
+    global _configfile
+    _configfile = os.abs.abspath(configfile)

onionr/core.py

@@ -1,7 +1,7 @@
 '''
     Onionr - P2P Microblogging Platform & Social network
 
-    Core Onionr library, useful for external programs. Handles peer processing and cryptography.
+    Core Onionr library, useful for external programs. Handles peer & data processing
 '''
 '''
     This program is free software: you can redistribute it and/or modify
@@ -17,12 +17,12 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
+import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
-from Crypto.Cipher import AES
+#from Crypto.Cipher import AES
-from Crypto import Random
+#from Crypto import Random
 import netcontroller
 
-import onionrutils
+import onionrutils, onionrcrypto, btc
 
 if sys.version_info < (3, 6):
     try:
@@ -34,106 +34,153 @@ if sys.version_info < (3, 6):
 class Core:
     def __init__(self):
         '''
-        Initialize Core Onionr library
+            Initialize Core Onionr library
         '''
         self.queueDB = 'data/queue.db'
         self.peerDB = 'data/peers.db'
-        self.ownPGPID = ''
         self.blockDB = 'data/blocks.db'
         self.blockDataLocation = 'data/blocks/'
-        self._utils = onionrutils.OnionrUtils(self)
+        self.addressDB = 'data/address.db'
 
         if not os.path.exists('data/'):
             os.mkdir('data/')
         if not os.path.exists('data/blocks/'):
             os.mkdir('data/blocks/')
-
         if not os.path.exists(self.blockDB):
             self.createBlockDB()
+            
+        self._utils = onionrutils.OnionrUtils(self)
+        # Initialize the crypto object
+        self._crypto = onionrcrypto.OnionrCrypto(self)
 
         return
 
-    def generateMainPGP(self, myID):
-        ''' Generate the main PGP key for our client. Should not be done often.
-        Uses own PGP home folder in the data/ directory. '''
-        # Generate main pgp key
-        gpg = gnupg.GPG(homedir='./data/pgp/')
-        input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
-        #input_data = gpg.gen_key_input(key_type="RSA", key_length=1024)
-        key = gpg.gen_key(input_data)
-        logger.info("Generating PGP key, this will take some time..")
-        while key.status != "key created":
-            time.sleep(0.5)
-            print(key.status)
-        logger.info("Finished generating PGP key")
-        # Write the key
-        myFingerpintFile = open('data/own-fingerprint.txt', 'w')
-        myFingerpintFile.write(key.fingerprint)
-        myFingerpintFile.close()
-        return
-
     def addPeer(self, peerID, name=''):
-        ''' Add a peer by their ID, with an optional name, to the peer database.'''
+        '''
-        ''' DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion. '''
+            Adds a public key to the key database (misleading function name)
+
+            DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion
+        '''
         # This function simply adds a peer to the DB
-        if not self._utils.validateID(peerID):
+        if not self._utils.validatePubKey(peerID):
             return False
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         t = (peerID, name, 'unknown')
-        c.execute('insert into peers (id, name, dateSeen) values(?, ?, ?);', t)
+        c.execute('INSERT INTO peers (id, name, dateSeen) VALUES(?, ?, ?);', t)
         conn.commit()
         conn.close()
         return True
 
+    def addAddress(self, address):
+        '''Add an address to the address database (only tor currently)'''
+        if self._utils.validateID(address):
+            conn = sqlite3.connect(self.addressDB)
+            c = conn.cursor()
+            t = (address, 1)
+            c.execute('INSERT INTO adders (address, type) VALUES(?, ?);', t)
+            conn.commit()
+            conn.close()
+            return True
+        else:
+            return False
+
+    def removeAddress(self, address):
+        '''Remove an address from the address database'''
+        if self._utils.validateID(address):
+            conn = sqlite3.connect(self.addressDB)
+            c = conn.cursor()
+            t = (address,)
+            c.execute('Delete from adders where address=?;', t)
+            conn.commit()
+            conn.close()
+            return True
+        else:
+            return False 
+
+    def createAddressDB(self):
+        '''
+            Generate the address database
+
+            types:
+                1: I2P b32 address
+                2: Tor v2 (like facebookcorewwwi.onion)
+                3: Tor v3
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        c.execute('''CREATE TABLE adders(
+            address text,
+            type int,
+            knownPeer text,
+            speed int,
+            success int,
+            DBHash text,
+            failure int
+            );
+        ''')
+        conn.commit()
+        conn.close()
+
     def createPeerDB(self):
         '''
-        Generate the peer sqlite3 database and populate it with the peers table.
+            Generate the peer sqlite3 database and populate it with the peers table.
         '''
         # generate the peer database
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
-        c.execute('''
+        c.execute('''CREATE TABLE peers(
-        create table peers(
+            ID text not null,
-        ID text not null,
+            name text,
-        name text,
+            adders text,
-        pgpKey text,
+            blockDBHash text,
-        hmacKey text,
+            forwardKey text,
-        blockDBHash text,
+            dateSeen not null,
-        forwardKey text,
+            bytesStored int,
-        dateSeen not null,
+            trust int);
-        bytesStored int,
-        trust int);
         ''')
         conn.commit()
         conn.close()
+        return
+
     def createBlockDB(self):
         '''
-        Create a database for blocks
+            Create a database for blocks
 
-        hash - the hash of a block
+            hash - the hash of a block
-        dateReceived - the date the block was recieved, not necessarily when it was created
+            dateReceived - the date the block was recieved, not necessarily when it was created
-        decrypted - if we can successfully decrypt the block (does not describe its current state)
+            decrypted - if we can successfully decrypt the block (does not describe its current state)
-        dataObtained - if the data has been obtained for the block
+            dataType - data type of the block
+            dataFound - if the data has been found for the block
+            dataSaved - if the data has been saved for the block
         '''
         if os.path.exists(self.blockDB):
             raise Exception("Block database already exists")
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
-        c.execute('''create table hashes(
+        c.execute('''CREATE TABLE hashes(
             hash text not null,
             dateReceived int,
             decrypted int,
+            dataType text,
             dataFound int,
-            dataSaved int
+            dataSaved int);
-        );
         ''')
         conn.commit()
         conn.close()
+
+        return
+
     def addToBlockDB(self, newHash, selfInsert=False):
-        '''add a hash value to the block db (should be in hex format)'''
+        '''
+            Add a hash value to the block db
+
+            Should be in hex format!
+        '''
         if not os.path.exists(self.blockDB):
             raise Exception('Block db does not exist')
+        if self._utils.hasBlock(newHash):
+            return
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
         currentTime = math.floor(time.time())
@@ -141,41 +188,57 @@ class Core:
             selfInsert = 1
         else:
             selfInsert = 0
-        data = (newHash, currentTime, 0, 0, selfInsert)
+        data = (newHash, currentTime, 0, '', 0, selfInsert)
-        c.execute('INSERT into hashes values(?, ?, ?, ?, ?);', data)
+        c.execute('INSERT INTO hashes VALUES(?, ?, ?, ?, ?, ?);', data)
         conn.commit()
         conn.close()
 
+        return
+
     def getData(self,hash):
-        '''simply return the data associated to a hash'''
+        '''
+            Simply return the data associated to a hash
+        '''
         try:
             dataFile = open(self.blockDataLocation + hash + '.dat')
             data = dataFile.read()
             dataFile.close()
         except FileNotFoundError:
             data = False
+
         return data
 
     def setData(self, data):
-        '''set the data assciated with a hash'''
+        '''
+            Set the data assciated with a hash
+        '''
         data = data.encode()
         hasher = hashlib.sha3_256()
         hasher.update(data)
         dataHash = hasher.hexdigest()
+        if type(dataHash) is bytes:
+            dataHash = dataHash.decode()
         blockFileName = self.blockDataLocation + dataHash + '.dat'
         if os.path.exists(blockFileName):
-            raise Exception("Data is already set for " + dataHash)
+            pass # TODO: properly check if block is already saved elsewhere
+            #raise Exception("Data is already set for " + dataHash)
         else:
             blockFile = open(blockFileName, 'w')
             blockFile.write(data.decode())
             blockFile.close()
+
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        c.execute("UPDATE hashes SET dataSaved=1 WHERE hash = '" + dataHash + "';")
+        conn.commit()
+        conn.close()
+
         return dataHash
 
     def dataDirEncrypt(self, password):
         '''
-        Encrypt the data directory on Onionr shutdown
+            Encrypt the data directory on Onionr shutdown
         '''
-        # Encrypt data directory (don't delete it in this function)
         if os.path.exists('data.tar'):
             os.remove('data.tar')
         tar = tarfile.open("data.tar", "w")
@@ -186,12 +249,13 @@ class Core:
         encrypted = simplecrypt.encrypt(password, tarData)
         open('data-encrypted.dat', 'wb').write(encrypted)
         os.remove('data.tar')
+
         return
+
     def dataDirDecrypt(self, password):
         '''
-        Decrypt the data directory on startup
+            Decrypt the data directory on startup
         '''
-        # Decrypt data directory
         if not os.path.exists('data-encrypted.dat'):
             return (False, 'encrypted archive does not exist')
         data = open('data-encrypted.dat', 'rb').read()
@@ -204,13 +268,15 @@ class Core:
             tar = tarfile.open('data.tar')
             tar.extractall()
             tar.close()
+
         return (True, '')
+
     def daemonQueue(self):
         '''
-        Gives commands to the communication proccess/daemon by reading an sqlite3 database
+            Gives commands to the communication proccess/daemon by reading an sqlite3 database
+
+            This function intended to be used by the client. Queue to exchange data between "client" and server.
         '''
-        # This function intended to be used by the client
-        # Queue to exchange data between "client" and server.
         retData = False
         if not os.path.exists(self.queueDB):
             conn = sqlite3.connect(self.queueDB)
@@ -226,7 +292,7 @@ class Core:
                 retData = row
                 break
             if retData != False:
-                c.execute('delete from commands where id = ?', (retData[3],))
+                c.execute('DELETE FROM commands WHERE id=?;', (retData[3],))
         conn.commit()
         conn.close()
 
@@ -234,19 +300,23 @@ class Core:
 
     def daemonQueueAdd(self, command, data=''):
         '''
-        Add a command to the daemon queue, used by the communication daemon (communicator.py)
+            Add a command to the daemon queue, used by the communication daemon (communicator.py)
         '''
         # Intended to be used by the web server
         date = math.floor(time.time())
         conn = sqlite3.connect(self.queueDB)
         c = conn.cursor()
         t = (command, data, date)
-        c.execute('INSERT into commands (command, data, date) values (?, ?, ?)', t)
+        c.execute('INSERT INTO commands (command, data, date) VALUES(?, ?, ?)', t)
         conn.commit()
         conn.close()
+
         return
+
     def clearDaemonQueue(self):
-        '''clear the daemon queue (somewhat dangerousous)'''
+        '''
+            Clear the daemon queue (somewhat dangerous)
+        '''
         conn = sqlite3.connect(self.queueDB)
         c = conn.cursor()
         try:
@@ -256,58 +326,59 @@ class Core:
             pass
         conn.close()
 
-    def generateHMAC(self):
+        return
+    
+    def listAdders(self, randomOrder=True, i2p=True):
         '''
-        generate and return an HMAC key
+            Return a list of addresses
         '''
-        key = base64.b64encode(os.urandom(32))
+        conn = sqlite3.connect(self.addressDB)
-        return key
+        c = conn.cursor()
+        if randomOrder:
+            addresses = c.execute('SELECT * FROM adders ORDER BY RANDOM();')
+        else:
+            addresses = c.execute('SELECT * FROM adders;')
+        addressList = []
+        for i in addresses:
+            addressList.append(i[0])
+        conn.close()
+        return addressList
 
     def listPeers(self, randomOrder=True):
-        '''Return a list of peers
+        '''
+            Return a list of public keys (misleading function name)
 
-        randomOrder determines if the list should be in a random order
+            randomOrder determines if the list should be in a random order
         '''
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         if randomOrder:
-            peers = c.execute('SELECT * FROM peers order by RANDOM();')
+            peers = c.execute('SELECT * FROM peers ORDER BY RANDOM();')
         else:
             peers = c.execute('SELECT * FROM peers;')
         peerList = []
         for i in peers:
-            peerList.append(i[0])
+            peerList.append(i[2])
         conn.close()
+
         return peerList
 
-    def processBlocks(self):
-        '''
-        Work with the block database and download any missing blocks
-        This is meant to be called from the communicator daemon on its timer.
-        '''
-        for i in self.getBlockList(True).split("\n"):
-            if i != "":
-                print('UNSAVED BLOCK:', i)
-        return
     def getPeerInfo(self, peer, info):
         '''
-        get info about a peer
+            Get info about a peer from their database entry
 
-        id text             0
+            id text             0
-        name text,          1
+            name text,          1
-        pgpKey text,        2
+            adders text,        2
-        hmacKey text,       3
+            forwardKey text,    3
-        blockDBHash text,   4
+            dateSeen not null,  4
-        forwardKey text,    5
+            bytesStored int,    5
-        dateSeen not null,  7
+            trust int           6
-        bytesStored int,    8
-        trust int           9
         '''
-        # Lookup something about a peer from their database entry
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         command = (peer,)
-        infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
+        infoNumbers = {'id': 0, 'name': 1, 'adders': 2, 'forwardKey': 3, 'dateSeen': 4, 'bytesStored': 5, 'trust': 6}
         info = infoNumbers[info]
         iterCount = 0
         retVal = ''
@@ -319,28 +390,109 @@ class Core:
                 else:
                     iterCount += 1
         conn.close()
+
         return retVal
+
     def setPeerInfo(self, peer, key, data):
-        '''update a peer for a key'''
+        '''
+            Update a peer for a key
+        '''
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         command = (data, peer)
         # TODO: validate key on whitelist
-
+        if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
-        c.execute('UPDATE peers SET ' + key + ' = ? where id=?', command)
+            raise Exception("Got invalid database key when setting peer info")
+        c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
         conn.commit()
         conn.close()
+        return
+
+    def getAddressInfo(self, address, info):
+        '''
+            Get info about an address from its database entry
+
+            address text, 0
+            type int, 1
+            knownPeer text, 2
+            speed int, 3
+            success int, 4
+            DBHash text, 5
+            failure int 6
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        command = (address,)
+        infoNumbers = {'address': 0, 'type': 1, 'knownPeer': 2, 'speed': 3, 'success': 4, 'DBHash': 5, 'failure': 6}
+        info = infoNumbers[info]
+        iterCount = 0
+        retVal = ''
+        for row in c.execute('SELECT * from adders where address=?;', command):
+            for i in row:
+                if iterCount == info:
+                    retVal = i
+                    break
+                else:
+                    iterCount += 1
+        conn.close()
+        return retVal
+
+    def setAddressInfo(self, address, key, data):
+        '''
+            Update an address for a key
+        '''
+        conn = sqlite3.connect(self.addressDB)
+        c = conn.cursor()
+        command = (data, address)
+        # TODO: validate key on whitelist
+        if key not in ('address', 'type', 'knownPeer', 'speed', 'success', 'DBHash', 'failure'):
+            raise Exception("Got invalid database key when setting address info")
+        c.execute('UPDATE adders SET ' + key + ' = ? WHERE address=?', command)
+        conn.commit()
+        conn.close()
+        return
 
     def getBlockList(self, unsaved=False):
-        '''get list of our blocks'''
+        '''
+            Get list of our blocks
+        '''
         conn = sqlite3.connect(self.blockDB)
         c = conn.cursor()
         retData = ''
         if unsaved:
-            execute = 'SELECT hash FROM hashes where dataSaved != 1;'
+            execute = 'SELECT hash FROM hashes WHERE dataSaved != 1;'
         else:
             execute = 'SELECT hash FROM hashes;'
         for row in c.execute(execute):
             for i in row:
                 retData += i + "\n"
+
         return retData
+
+    def getBlocksByType(self, blockType):
+        '''
+            Returns a list of blocks by the type
+        '''
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        retData = ''
+        execute = 'SELECT hash FROM hashes WHERE dataType=?;'
+        args = (blockType,)
+        for row in c.execute(execute, args):
+            for i in row:
+                retData += i + "\n"
+
+        return retData.split('\n')
+
+    def setBlockType(self, hash, blockType):
+        '''
+            Sets the type of block
+        '''
+
+        conn = sqlite3.connect(self.blockDB)
+        c = conn.cursor()
+        c.execute("UPDATE hashes SET dataType='" + blockType + "' WHERE hash = '" + hash + "';")
+        conn.commit()
+        conn.close()
+
+        return

onionr/gui.py

@@ -13,4 +13,58 @@
 
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
-'''
+'''
+from tkinter import *
+import os, sqlite3, core
+class OnionrGUI:
+    def __init__(self, myCore):
+        self.root = Tk()
+        self.myCore = myCore # onionr core
+        self.root.title("PyOnionr")
+
+        w = Label(self.root, text="Onionr", width=10)
+        w.config(font=("Sans-Serif", 22))
+        w.pack()
+        scrollbar = Scrollbar(self.root)
+        scrollbar.pack(side=RIGHT, fill=Y)
+
+        self.listedBlocks = []
+
+        idText = open('./data/hs/hostname', 'r').read()
+        idLabel = Label(self.root, text="ID: " + idText)
+        idLabel.pack(pady=5)
+
+        self.sendEntry = Entry(self.root)
+        sendBtn = Button(self.root, text='Send Message', command=self.sendMessage)
+        self.sendEntry.pack()
+        sendBtn.pack()
+
+        self.listbox = Listbox(self.root, yscrollcommand=scrollbar.set, height=15)
+
+        #listbox.insert(END, str(i))
+        self.listbox.pack(fill=BOTH)
+
+        scrollbar.config(command=self.listbox.yview)
+        self.root.after(2000, self.update)
+        self.root.mainloop() 
+
+    def sendMessage(self):
+        messageToAdd = '-txt-' + self.sendEntry.get()
+        addedHash = self.myCore.setData(messageToAdd)
+        self.myCore.addToBlockDB(addedHash, selfInsert=True)
+        self.myCore.setBlockType(addedHash, 'txt')
+        self.sendEntry.delete(0, END)
+
+    def update(self):
+        for i in self.myCore.getBlocksByType('txt'):
+            if i.strip() == '' or i in self.listedBlocks:
+                continue
+            blockFile = open('./data/blocks/' + i + '.dat')
+            self.listbox.insert(END, str(blockFile.read().replace('-txt-', '')))
+            blockFile.close()
+            self.listedBlocks.append(i)
+            self.listbox.see(END)
+        blocksList = os.listdir('./data/blocks/') # dir is your directory path
+        number_blocks = len(blocksList)
+
+        self.root.after(10000, self.update)

onionr/logger.py

@@ -78,60 +78,82 @@ _type = OUTPUT_TO_CONSOLE | USE_ANSI # the default settings for logging
 _level = LEVEL_DEBUG # the lowest level to log
 _outputfile = './output.log' # the file to log to
 
-'''
-    Set the settings for the logger using bitwise operators
-'''
 def set_settings(type):
+    '''
+        Set the settings for the logger using bitwise operators
+    '''
+
     global _type
     _type = type
 
-'''
-    Get settings from the logger
-'''
 def get_settings():
+    '''
+        Get settings from the logger
+    '''
+
     return _type
 
-'''
-    Set the lowest log level to output
-'''
 def set_level(level):
+    '''
+        Set the lowest log level to output
+    '''
+
     global _level
     _level = level
 
-'''
-    Get the lowest log level currently being outputted
-'''
 def get_level():
+    '''
+        Get the lowest log level currently being outputted
+    '''
+
     return _level
 
-'''
+def set_file(outputfile):
-    Outputs raw data to console without formatting
+    '''
-'''
+        Set the file to output to, if enabled
+    '''
+
+    global _outputfile
+    _outputfile = outputfile
+
+def get_file():
+    '''
+        Get the file to output to
+    '''
+
+    return _outputfile
+
 def raw(data):
+    '''
+        Outputs raw data to console without formatting
+    '''
+
     if get_settings() & OUTPUT_TO_CONSOLE:
         print(data)
     if get_settings() & OUTPUT_TO_FILE:
         with open(_outputfile, "a+") as f:
             f.write(colors.filter(data) + '\n')
 
-'''
-    Logs the data
-    prefix : The prefix to the output
-    data   : The actual data to output
-    color  : The color to output before the data
-'''
 def log(prefix, data, color = ''):
+    '''
+        Logs the data
+        prefix : The prefix to the output
+        data   : The actual data to output
+        color  : The color to output before the data
+    '''
+
     output = colors.reset + str(color) + '[' + colors.bold + str(prefix) + colors.reset + str(color) + '] ' + str(data) + colors.reset
     if not get_settings() & USE_ANSI:
         output = colors.filter(output)
 
     raw(output)
 
-'''
+def readline(message = ''):
-    Takes in input from the console, not stored in logs
+    '''
-    message: The message to display before taking input
+        Takes in input from the console, not stored in logs
-'''
+        message: The message to display before taking input
-def input(message = 'Enter input: '):
+    '''
+
     color = colors.fg.green + colors.bold
     output = colors.reset + str(color) + '... ' + colors.reset + str(message) + colors.reset
 
@@ -139,14 +161,16 @@ def input(message = 'Enter input: '):
         output = colors.filter(output)
 
     sys.stdout.write(output)
-    return raw_input()
 
-'''
+    return input()
-    Displays an "Are you sure" message, returns True for Y and False for N
+
-    message: The confirmation message, use %s for (y/n)
-    default: which to prefer-- y or n
-'''
 def confirm(default = 'y', message = 'Are you sure %s? '):
+    '''
+        Displays an "Are you sure" message, returns True for Y and False for N
+        message: The confirmation message, use %s for (y/n)
+        default: which to prefer-- y or n
+    '''
+
     color = colors.fg.green + colors.bold
 
     default = default.lower()
@@ -163,7 +187,8 @@ def confirm(default = 'y', message = 'Are you sure %s? '):
         output = colors.filter(output)
 
     sys.stdout.write(output.replace('%s', confirm))
-    inp = raw_input().lower()
+
+    inp = input().lower()
 
     if 'y' in inp:
         return True

onionr/netcontroller.py

@@ -17,11 +17,14 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
+
 import subprocess, os, random, sys, logger, time, signal
+
 class NetController:
-    '''NetController
-    This class handles hidden service setup on Tor and I2P
     '''
+        This class handles hidden service setup on Tor and I2P
+    '''
+
     def __init__(self, hsPort):
         self.torConfigLocation = 'data/torrc'
         self.readyState = False
@@ -30,15 +33,20 @@ class NetController:
         self._torInstnace = ''
         self.myID = ''
         '''
-        if os.path.exists(self.torConfigLocation):
+            if os.path.exists(self.torConfigLocation):
-            torrc = open(self.torConfigLocation, 'r')
+                torrc = open(self.torConfigLocation, 'r')
-            if not str(self.hsPort) in torrc.read():
+                if not str(self.hsPort) in torrc.read():
-                os.remove(self.torConfigLocation)
+                    os.remove(self.torConfigLocation)
-            torrc.close()
+                torrc.close()
         '''
+
         return
+
     def generateTorrc(self):
-        '''generate a torrc file for our tor instance'''
+        '''
+            Generate a torrc file for our tor instance
+        '''
+
         if os.path.exists(self.torConfigLocation):
             os.remove(self.torConfigLocation)
         torrcData = '''SocksPort ''' + str(self.socksPort) + '''
@@ -48,50 +56,83 @@ HiddenServicePort 80 127.0.0.1:''' + str(self.hsPort) + '''
         torrc = open(self.torConfigLocation, 'w')
         torrc.write(torrcData)
         torrc.close()
+
         return
 
     def startTor(self):
-        '''Start Tor with onion service on port 80 & socks proxy on random port
         '''
+            Start Tor with onion service on port 80 & socks proxy on random port
+        '''
+
         self.generateTorrc()
+
         if os.path.exists('./tor'):
             torBinary = './tor'
+        elif os.path.exists('/usr/bin/tor'):
+            torBinary = '/usr/bin/tor'
         else:
             torBinary = 'tor'
+
         try:
             tor = subprocess.Popen([torBinary, '-f', self.torConfigLocation], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         except FileNotFoundError:
             logger.fatal("Tor was not found in your path or the Onionr directory. Please install Tor and try again.")
             sys.exit(1)
+        else:
+            # Test Tor Version
+            torVersion = subprocess.Popen([torBinary, '--version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+            for line in iter(torVersion.stdout.readline, b''):
+                if 'Tor 0.2.' in line.decode():
+                    logger.warn("Running 0.2.x Tor series, no support for v3 onion peers")
+                    break
+            torVersion.kill()
+
         # wait for tor to get to 100% bootstrap
         for line in iter(tor.stdout.readline, b''):
             if 'Bootstrapped 100%: Done' in line.decode():
                 break
             elif 'Opening Socks listener' in line.decode():
-                logger.debug(line.decode())
+                logger.debug(line.decode().replace('\n', ''))
         else:
             logger.fatal('Failed to start Tor. Try killing any other Tor processes owned by this user.')
             return False
+
         logger.info('Finished starting Tor')
         self.readyState = True
+
         myID = open('data/hs/hostname', 'r')
-        self.myID = myID.read()
+        self.myID = myID.read().replace('\n', '')
         myID.close()
+
         torPidFile = open('data/torPid.txt', 'w')
         torPidFile.write(str(tor.pid))
         torPidFile.close()
+
         return True
+
     def killTor(self):
-        '''properly kill tor based on pid saved to file'''
+        '''
+            Properly kill tor based on pid saved to file
+        '''
+
         try:
             pid = open('data/torPid.txt', 'r')
             pidN = pid.read()
             pid.close()
         except FileNotFoundError:
             return
+
         try:
             int(pidN)
         except:
             return
-        os.kill(int(pidN), signal.SIGTERM)
+
-        os.remove('data/torPid.txt')
+        try:
+            os.kill(int(pidN), signal.SIGTERM)
+            os.remove('data/torPid.txt')
+        except ProcessLookupError:
+            pass
+        except FileNotFoundError:
+            pass
+
+        return

onionr/onionr.py

@@ -20,8 +20,8 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sys, os, configparser, base64, random, getpass, shutil, subprocess, requests, time, logger
+import sys, os, base64, random, getpass, shutil, subprocess, requests, time, platform
-import gui, api, core
+import api, core, gui, config, logger, onionrplugins as plugins
 from onionrutils import OnionrUtils
 from netcontroller import NetController
 
@@ -30,20 +30,44 @@ try:
 except ImportError:
     raise Exception("You need the PySocks module (for use with socks5 proxy to use Tor)")
 
-class Onionr:
+ONIONR_TAGLINE = 'Anonymous P2P Platform - GPLv3 - onionr.voidnet.tech'
-    def __init__(self):
+ONIONR_VERSION = '0.0.0' # for debugging and stuff
-        '''Main Onionr class. This is for the CLI program, and does not handle much of the logic.
+API_VERSION = '1' # increments of 1; only change when something fundemental about how the API works changes. This way other nodes knows how to communicate without learning too much information about you.
-        In general, external programs and plugins should not use this class.
 
+class Onionr:
+    cmds = {}
+    cmdhelp = {}
+
+    def __init__(self):
         '''
+            Main Onionr class. This is for the CLI program, and does not handle much of the logic.
+            In general, external programs and plugins should not use this class.
+        '''
+
         try:
             os.chdir(sys.path[0])
         except FileNotFoundError:
             pass
-        if os.path.exists('dev-enabled'):
+
+        # Load global configuration data
+
+        exists = os.path.exists(config.get_config_file())
+        config.set_config({'devmode': True, 'log': {'file': {'output': True, 'path': 'data/output.log'}, 'console': {'output': True, 'color': True}}}) # this is the default config, it will be overwritten if a config file already exists. Else, it saves it
+        config.reload() # this will read the configuration file into memory
+
+        settings = 0b000
+        if config.get('log', {'console': {'color': True}})['console']['color']:
+            settings = settings | logger.USE_ANSI
+        if config.get('log', {'console': {'output': True}})['console']['output']:
+            settings = settings | logger.OUTPUT_TO_CONSOLE
+        if config.get('log', {'file': {'output': True}})['file']['output']:
+            settings = settings | logger.OUTPUT_TO_FILE
+            logger.set_file(config.get('log', {'file': {'path': 'data/output.log'}})['file']['path'])
+        logger.set_settings(settings)
+
+        if config.get('devmode', True):
             self._developmentMode = True
             logger.set_level(logger.LEVEL_DEBUG)
-            logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
         else:
             self._developmentMode = False
             logger.set_level(logger.LEVEL_INFO)
@@ -51,7 +75,7 @@ class Onionr:
         self.onionrCore = core.Core()
         self.onionrUtils = OnionrUtils(self.onionrCore)
 
-        # Get configuration and Handle commands
+        # Handle commands
 
         self.debug = False # Whole application debugging
 
@@ -69,15 +93,15 @@ class Onionr:
                 os.mkdir('data/')
                 os.mkdir('data/blocks/')
 
-        if not os.path.exists('data/peers.db'):
+        if not os.path.exists(self.onionrCore.peerDB):
             self.onionrCore.createPeerDB()
             pass
+        if not os.path.exists(self.onionrCore.addressDB):
+            self.onionrCore.createAddressDB()
 
         # Get configuration
-        self.config = configparser.ConfigParser()
+
-        if os.path.exists('data/config.ini'):
+        if not exists:
-            self.config.read('data/config.ini')
-        else:
             # Generate default config
             # Hostname should only be set if different from 127.x.x.x. Important for DNS rebinding attack prevention.
             if self.debug:
@@ -87,78 +111,364 @@ class Onionr:
                     randomPort = random.randint(1024, 65535)
                     if self.onionrUtils.checkPort(randomPort):
                         break
-            self.config['CLIENT'] = {'CLIENT HMAC': base64.b64encode(os.urandom(32)).decode('utf-8'), 'PORT': randomPort, 'API VERSION': '0.0.0'}
+            config.set('client', {'participate': 'true', 'client_hmac': base64.b64encode(os.urandom(32)).decode('utf-8'), 'port': randomPort, 'api_version': API_VERSION}, True)
-            with open('data/config.ini', 'w') as configfile:
+
-                self.config.write(configfile)
+        self.cmds = {
+            '': self.showHelpSuggestion,
+            'help': self.showHelp,
+            'version': self.version,
+            'config': self.configure,
+            'start': self.start,
+            'stop': self.killDaemon,
+            'stats': self.showStats,
+
+            'enable-plugin': self.enablePlugin,
+            'enplugin': self.enablePlugin,
+            'enableplugin': self.enablePlugin,
+            'enmod': self.enablePlugin,
+            'disable-plugin': self.disablePlugin,
+            'displugin': self.disablePlugin,
+            'disableplugin': self.disablePlugin,
+            'dismod': self.disablePlugin,
+            'reload-plugin': self.reloadPlugin,
+            'reloadplugin': self.reloadPlugin,
+            'reload-plugins': self.reloadPlugin,
+            'reloadplugins': self.reloadPlugin,
+
+            'listpeers': self.listPeers,
+            'list-peers': self.listPeers,
+
+            'addmsg': self.addMessage,
+            'addmessage': self.addMessage,
+            'add-msg': self.addMessage,
+            'add-message': self.addMessage,
+            'pm': self.sendEncrypt,
+
+            'gui': self.openGUI,
+
+            'addpeer': self.addPeer,
+            'add-peer': self.addPeer,
+            'add-address': self.addAddress,
+            'addaddress': self.addAddress,
+
+            'connect': self.addAddress
+        }
+
+        self.cmdhelp = {
+            'help': 'Displays this Onionr help menu',
+            'version': 'Displays the Onionr version',
+            'config': 'Configures something and adds it to the file',
+            'start': 'Starts the Onionr daemon',
+            'stop': 'Stops the Onionr daemon',
+            'stats': 'Displays node statistics',
+            'enable-plugin': 'Enables and starts a plugin',
+            'disable-plugin': 'Disables and stops a plugin',
+            'reload-plugin': 'Reloads a plugin',
+            'list-peers': 'Displays a list of peers',
+            'add-peer': 'Adds a peer (?)',
+            'add-msg': 'Broadcasts a message to the Onionr network',
+            'pm': 'Adds a private message to block',
+            'gui': 'Opens a graphical interface for Onionr'
+        }
+
         command = ''
         try:
             command = sys.argv[1].lower()
         except IndexError:
             command = ''
         finally:
-            if command == 'start':
+            self.execute(command)
-                if os.path.exists('.onionr-lock'):
-                    logger.fatal('Cannot start. Daemon is already running, or it did not exit cleanly.\n(if you are sure that there is not a daemon running, delete .onionr-lock & try again).')
-                else:
-                    if not self.debug and not self._developmentMode:
-                        lockFile = open('.onionr-lock', 'w')
-                        lockFile.write('')
-                        lockFile.close()
-                    self.daemon()
-                    if not self.debug and not self._developmentMode:
-                        os.remove('.onionr-lock')
-            elif command == 'stop':
-                self.killDaemon()
-            elif command in ('addmsg', 'addmessage'):
-                while True:
-                    messageToAdd = input('Broadcast message to network: ')
-                    if len(messageToAdd) >= 1:
-                        break
-                addedHash = self.onionrCore.setData(messageToAdd)
-                self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
-            elif command == 'stats':
-                self.showStats()
-            elif command == 'help' or command == '--help':
-                self.showHelp()
-            elif command == '':
-                logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
-            else:
-                logger.error('Invalid command.')
 
         if not self._developmentMode:
             encryptionPassword = self.onionrUtils.getPassword('Enter password to encrypt directory: ')
             self.onionrCore.dataDirEncrypt(encryptionPassword)
             shutil.rmtree('data/')
+
         return
-    def daemon(self):
+
-        ''' Start the Onionr communication daemon
+    '''
+        THIS SECTION HANDLES THE COMMANDS
+    '''
+
+    def getCommands(self):
+        return self.cmds
+
+    def getHelp(self):
+        return self.cmdhelp
+
+    def addCommand(self, command, function):
+        cmds[str(command).lower()] = function
+
+    def addHelp(self, command, description):
+        cmdhelp[str(command).lower()] = str(description)
+
+    def configure(self):
         '''
+            Displays something from the configuration file, or sets it
+        '''
+
+        if len(sys.argv) >= 4:
+            config.reload()
+            config.set(sys.argv[2], sys.argv[3], True)
+            logger.debug('Configuration file updated.')
+        elif len(sys.argv) >= 3:
+            config.reload()
+            logger.info(logger.colors.bold + sys.argv[2] + ': ' + logger.colors.reset + str(config.get(sys.argv[2], logger.colors.fg.red + 'Not set.')))
+        else:
+            logger.info(logger.colors.bold + 'Get a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key>')
+            logger.info(logger.colors.bold + 'Set a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key> <value>')
+
+
+    def execute(self, argument):
+        '''
+            Executes a command
+        '''
+        argument = argument[argument.startswith('--') and len('--'):] # remove -- if it starts with it
+
+        # define commands
+        commands = self.getCommands()
+
+        command = commands.get(argument, self.notFound)
+        command()
+
+    '''
+        THIS SECTION DEFINES THE COMMANDS
+    '''
+
+    def version(self, verbosity=5):
+        '''
+            Displays the Onionr version
+        '''
+        logger.info('Onionr ' + ONIONR_VERSION + ' (' + platform.machine() + ') - API v' + API_VERSION)
+        if verbosity >= 1:
+            logger.info(ONIONR_TAGLINE)
+        if verbosity >= 2:
+            logger.info('Running on ' + platform.platform() + ' ' + platform.release())
+
+    def sendEncrypt(self):
+        '''
+            Create a private message and send it
+        '''
+
+        while True:
+            try:
+                peer = logger.readline('Peer to send to: ')
+            except KeyboardInterrupt:
+                break
+            else:
+                if self.onionrUtils.validateID(peer):
+                    break
+                else:
+                    logger.error('Invalid peer ID')
+        else:
+            try:
+                message = logger.readline("Enter a message: ")
+            except KeyboardInterrupt:
+                pass
+            else:
+                logger.info("Sending message to " + peer)
+                self.onionrUtils.sendPM(peer, message)
+
+
+    def openGUI(self):
+        '''
+            Opens a graphical interface for Onionr
+        '''
+
+        gui.OnionrGUI(self.onionrCore)
+
+    def listPeers(self):
+        '''
+            Displays a list of peers (?)
+        '''
+
+        logger.info('Peer list:\n')
+        for i in self.onionrCore.listPeers():
+            logger.info(i)
+
+    def addPeer(self):
+        '''
+            Adds a peer (?)
+        '''
+
+        try:
+            newPeer = sys.argv[2]
+        except:
+            pass
+        else:
+            logger.info("Adding peer: " + logger.colors.underline + newPeer)
+            self.onionrCore.addPeer(newPeer)
+
+        return
+
+    def addAddress(self):
+        '''Adds a Onionr node address'''
+        try:
+            newAddress = sys.argv[2]
+        except:
+            pass
+        else:
+            logger.info("Adding address: " + logger.colors.underline + newAddress)
+            if self.onionrCore.addAddress(newAddress):
+                logger.info("Successfully added address")
+            else:
+                logger.warn("Unable to add address")
+
+        return
+
+    def addMessage(self):
+        '''
+            Broadcasts a message to the Onionr network
+        '''
+
+        while True:
+            messageToAdd = '-txt-' + logger.readline('Broadcast message to network: ')
+            if len(messageToAdd) >= 1:
+                break
+
+        addedHash = self.onionrCore.setData(messageToAdd)
+        self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
+        self.onionrCore.setBlockType(addedHash, 'txt')
+
+        return
+
+    def enablePlugin(self):
+        '''
+            Enables and starts the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Enabling plugin \"' + plugin_name + '\"...')
+            plugins.enable(plugin_name)
+        else:
+            logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
+
+        return
+
+    def disablePlugin(self):
+        '''
+            Disables and stops the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Disabling plugin \"' + plugin_name + '\"...')
+            plugins.disable(plugin_name)
+        else:
+            logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
+
+        return
+
+    def reloadPlugin(self):
+        '''
+            Reloads (stops and starts) all plugins, or the given plugin
+        '''
+
+        if len(sys.argv) >= 3:
+            plugin_name = sys.argv[2]
+            logger.info('Reloading plugin \"' + plugin_name + '\"...')
+            plugins.stop(plugin_name)
+            plugins.start(plugin_name)
+        else:
+            logger.info('Reloading all plugins...')
+            plugins.reload()
+
+        return
+
+    def notFound(self):
+        '''
+            Displays a "command not found" message
+        '''
+
+        logger.error('Command not found.')
+
+    def showHelpSuggestion(self):
+        '''
+            Displays a message suggesting help
+        '''
+
+        logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
+
+    def start(self):
+        '''
+            Starts the Onionr daemon
+        '''
+
+        if os.path.exists('.onionr-lock'):
+            logger.fatal('Cannot start. Daemon is already running, or it did not exit cleanly.\n(if you are sure that there is not a daemon running, delete .onionr-lock & try again).')
+        else:
+            if not self.debug and not self._developmentMode:
+                lockFile = open('.onionr-lock', 'w')
+                lockFile.write('')
+                lockFile.close()
+            self.daemon()
+            if not self.debug and not self._developmentMode:
+                os.remove('.onionr-lock')
+
+    def daemon(self):
+        '''
+            Starts the Onionr communication daemon
+        '''
+
         if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
-            net = NetController(self.config['CLIENT']['PORT'])
+            if self._developmentMode:
+                logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
+            net = NetController(config.get('client')['port'])
             logger.info('Tor is starting...')
             if not net.startTor():
                 sys.exit(1)
             logger.info('Started Tor .onion service: ' + logger.colors.underline + net.myID)
+            logger.info('Our Public key: ' + self.onionrCore._crypto.pubKey)
             time.sleep(1)
             subprocess.Popen(["./communicator.py", "run", str(net.socksPort)])
             logger.debug('Started communicator')
-        api.API(self.config, self.debug)
+        api.API(self.debug)
+
         return
+
     def killDaemon(self):
-        '''Shutdown the Onionr Daemon'''
+        '''
+            Shutdown the Onionr daemon
+        '''
+
         logger.warn('Killing the running daemon')
-        net = NetController(self.config['CLIENT']['PORT'])
+        net = NetController(config.get('client')['port'])
         try:
             self.onionrUtils.localCommand('shutdown')
         except requests.exceptions.ConnectionError:
             pass
         self.onionrCore.daemonQueueAdd('shutdown')
         net.killTor()
+
         return
+
     def showStats(self):
-        '''Display statistics and exit'''
+        '''
+            Displays statistics and exits
+        '''
+
         return
-    def showHelp(self):
+
-        '''Show help for Onionr'''
+    def showHelp(self, command = None):
+        '''
+            Show help for Onionr
+        '''
+
+        helpmenu = self.getHelp()
+
+        if command is None and len(sys.argv) >= 3:
+            for cmd in sys.argv[2:]:
+                self.showHelp(cmd)
+        elif not command is None:
+            if command.lower() in helpmenu:
+                logger.info(logger.colors.bold + command  + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset +  helpmenu[command.lower()])
+            else:
+                logger.warn(logger.colors.bold + command  + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset + 'No help menu entry was found')
+        else:
+            self.version(0)
+            for command, helpmessage in helpmenu.items():
+                self.showHelp(command)
         return
-Onionr()
+
+Onionr()

onionr/onionrcrypto.py

@@ -0,0 +1,97 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file handles Onionr's cryptography.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+import nacl.signing, nacl.encoding, nacl.public, os
+
+class OnionrCrypto:
+    def __init__(self, coreInstance):
+        self._core = coreInstance
+        self._keyFile = 'data/keys.txt'
+        self.pubKey = None
+        self.privKey = None
+
+        # Load our own pub/priv Ed25519 keys, gen & save them if they don't exist
+        if os.path.exists(self._keyFile):
+            with open('data/keys.txt', 'r') as keys:
+                keys = keys.read().split(',')
+                self.pubKey = keys[0]
+                self.privKey = keys[1]
+        else:
+            keys = self.generatePubKey()
+            self.pubKey = keys[0]
+            self.privKey = keys[1]
+            with open(self._keyFile, 'w') as keyfile:
+                keyfile.write(self.pubKey + ',' + self.privKey)
+        return
+
+    def edVerify(self, data, key):
+        '''Verify signed data (combined in nacl) to an ed25519 key'''
+        key = nacl.signing.VerifyKey(key=key, encoder=nacl.encoding.Base32Encoder)
+        retData = ''
+        if encodeResult:
+            retData = key.verify(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
+        else:
+            retData = key.verify(data.encode())
+        return retData
+    
+    def edSign(self, data, key, encodeResult=False):
+        '''Ed25519 sign data'''
+        key = nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
+        retData = ''
+        if encodeResult:
+            retData = key.sign(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
+        else:
+            retData = key.sign(data.encode())
+        return retData
+
+    def pubKeyEncrypt(self, data, pubkey, anonymous=False):
+        '''Encrypt to a public key (Curve25519, taken from base32 Ed25519 pubkey)'''
+        retVal = ''
+        if self.privKey != None and not anonymous:
+            ownKey = nacl.signing.SigningKey(seed=self.privKey, encoder=nacl.encoding.Base32Encoder())
+            key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
+            ourBox = nacl.public.Box(ownKey, key)
+            retVal = ourBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
+        elif anonymous:
+            key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
+            anonBox = nacl.public.SealedBox(key)
+            retVal = anonBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
+        return retVal
+
+    def pubKeyDecrypt(self, data, peer):
+        '''pubkey decrypt (Curve25519, taken from Ed25519 pubkey)'''
+        return
+
+    def symmetricPeerEncrypt(self, data):
+        '''Salsa20 encrypt data to peer (with mac)'''
+        return
+
+    def symmetricPeerDecrypt(self, data, peer):
+        '''Salsa20 decrypt data from peer (with mac)'''
+        return
+    
+    def generateSymmetric(self, data, peer):
+        '''Generate symmetric key'''
+        return
+
+    def generatePubKey(self):
+        '''Generate a Ed25519 public key pair, return tuple of base64encoded pubkey, privkey'''
+        private_key = nacl.signing.SigningKey.generate()
+        public_key = private_key.verify_key.encode(encoder=nacl.encoding.Base32Encoder())
+        return (public_key.decode(), private_key.encode(encoder=nacl.encoding.Base32Encoder()).decode())

onionr/onionrevents.py

@@ -0,0 +1,53 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with configuration management.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import config, logger, onionrplugins as plugins
+
+def event(event_name, data = None, onionr = None):
+    '''
+        Calls an event on all plugins (if defined)
+    '''
+
+    for plugin in plugins.get_enabled_plugins():
+        try:
+            call(plugins.get_plugin(plugin), event_name, data, onionr)
+        except:
+            logger.warn('Event \"' + event_name + '\" failed for plugin \"' + plugin + '\".')
+
+def call(plugin, event_name, data = None, onionr = None):
+    '''
+        Calls an event on a plugin if one is defined
+    '''
+
+    if not plugin is None:
+        try:
+            attribute = 'on_' + str(event_name).lower()
+
+            # TODO: Use multithreading perhaps?
+            if hasattr(plugin, attribute):
+                logger.debug('Calling event ' + str(event_name))
+                getattr(plugin, attribute)(onionr, data)
+
+            return True
+        except:
+            logger.warn('Failed to call event ' + str(event_name) + ' on module.')
+            return False
+    else:
+        return True

onionr/onionrplugins.py

@@ -0,0 +1,234 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    This file deals with management of modules/plugins.
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+
+import os, re, importlib, config, logger
+import onionrevents as events
+
+_pluginsfolder = 'data/plugins/'
+_instances = dict()
+
+def reload(stop_event = True):
+    '''
+        Reloads all the plugins
+    '''
+
+    check()
+
+    try:
+        enabled_plugins = get_enabled_plugins()
+
+        if stop_event is True:
+            logger.debug('Reloading all plugins...')
+        else:
+            logger.debug('Loading all plugins...')
+
+        if stop_event is True:
+            for plugin in enabled_plugins:
+                stop(plugin)
+
+        for plugin in enabled_plugins:
+            start(plugin)
+
+        return True
+    except:
+        logger.error('Failed to reload plugins.')
+
+    return False
+
+
+def enable(name, start_event = True):
+    '''
+        Enables a plugin
+    '''
+
+    check()
+
+    if exists(name):
+        enabled_plugins = get_enabled_plugins()
+        enabled_plugins.append(name)
+        config_plugins = config.get('plugins')
+        config_plugins['enabled'] = enabled_plugins
+        config.set('plugins', config_plugins, True)
+
+        events.call(get_plugin(name), 'enable')
+
+        if start_event is True:
+            start(name)
+
+        return True
+    else:
+        logger.error('Failed to enable plugin \"' + name + '\", disabling plugin.')
+        disable(name)
+
+        return False
+
+
+def disable(name, stop_event = True):
+    '''
+        Disables a plugin
+    '''
+
+    check()
+
+    if is_enabled(name):
+        enabled_plugins = get_enabled_plugins()
+        enabled_plugins.remove(name)
+        config_plugins = config.get('plugins')
+        config_plugins['enabled'] = enabled_plugins
+        config.set('plugins', config_plugins, True)
+
+    if exists(name):
+        events.call(get_plugin(name), 'disable')
+
+        if stop_event is True:
+            stop(name)
+
+def start(name):
+    '''
+        Starts the plugin
+    '''
+
+    check()
+
+    if exists(name):
+        try:
+            plugin = get_plugin(name)
+
+            if plugin is None:
+                raise Exception('Failed to import module.')
+            else:
+                events.call(plugin, 'start')
+
+            return plugin
+        except:
+            logger.error('Failed to start module \"' + name + '\".')
+    else:
+        logger.error('Failed to start nonexistant module \"' + name + '\".')
+
+    return None
+
+def stop(name):
+    '''
+        Stops the plugin
+    '''
+
+    check()
+
+    if exists(name):
+        try:
+            plugin = get_plugin(name)
+
+            if plugin is None:
+                raise Exception('Failed to import module.')
+            else:
+                events.call(plugin, 'stop')
+
+            return plugin
+        except:
+            logger.error('Failed to stop module \"' + name + '\".')
+    else:
+        logger.error('Failed to stop nonexistant module \"' + name + '\".')
+
+    return None
+
+def get_plugin(name):
+    '''
+        Returns the instance of a module
+    '''
+
+    check()
+
+    if str(name).lower() in _instances:
+        return _instances[str(name).lower()]
+    else:
+        _instances[str(name).lower()] = importlib.import_module(get_plugins_folder(name, False).replace('/', '.') + 'main')
+        return get_plugin(name)
+
+def get_plugins():
+    '''
+        Returns a list of plugins (deprecated)
+    '''
+
+    return _instances
+
+def exists(name):
+    '''
+        Return value indicates whether or not the plugin exists
+    '''
+
+    return os.path.isdir(get_plugins_folder(str(name).lower()))
+
+def get_enabled_plugins():
+    '''
+        Returns a list of the enabled plugins
+    '''
+
+    check()
+
+    config.reload()
+
+    return config.get('plugins')['enabled']
+
+def is_enabled(name):
+    '''
+        Return value indicates whether or not the plugin is enabled
+    '''
+
+    return name in get_enabled_plugins()
+
+def get_plugins_folder(name = None, absolute = True):
+    '''
+        Returns the path to the plugins folder
+    '''
+
+    path = ''
+
+    if name is None:
+        path = _pluginsfolder
+    else:
+        # only allow alphanumeric characters
+        path = _pluginsfolder + re.sub('[^0-9a-zA-Z]+', '', str(name).lower()) + '/'
+
+    if absolute is True:
+        path = os.path.abspath(path)
+
+    return path
+
+def check():
+    '''
+        Checks to make sure files exist
+    '''
+
+    config.reload()
+
+    if not config.is_set('plugins'):
+        logger.debug('Generating plugin config data...')
+        config.set('plugins', {'enabled': []}, True)
+
+    if not os.path.exists(os.path.dirname(get_plugins_folder())):
+        logger.debug('Generating plugin data folder...')
+        os.makedirs(os.path.dirname(get_plugins_folder()))
+
+    if not exists('test'):
+        os.makedirs(get_plugins_folder('test'))
+        with open(get_plugins_folder('test') + '/main.py', 'a') as main:
+            main.write("print('Running')\n\ndef on_test(onionr = None, data = None):\n    print('received test event!')\n    return True\n\ndef on_start(onionr = None, data = None):\n    print('start event called')\n\ndef on_stop(onionr = None, data = None):\n    print('stop event called')\n\ndef on_enable(onionr = None, data = None):\n    print('enable event called')\n\ndef on_disable(onionr = None, data = None):\n    print('disable event called')\n")
+        enable('test')
+    return

onionr/onionrproofs.py

@@ -0,0 +1,86 @@
+'''
+    Onionr - P2P Microblogging Platform & Social network
+
+    Proof of work module
+'''
+'''
+    This program is free software: you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation, either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
+'''
+import nacl.encoding, nacl.hash, nacl.utils, time, math, threading, binascii, logger
+import btc
+class POW:
+    def pow(self, reporting=False):
+        startTime = math.floor(time.time())
+        self.hashing = True
+        self.reporting = reporting
+        iFound = False # if current thread is the one that found the answer
+        answer = ''
+        heartbeat = 200000
+        hbCount = 0
+        blockCheck = 300000 # How often the hasher should check if the bitcoin block is updated (slows hashing but prevents less wasted work)
+        blockCheckCount = 0
+        block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
+        while self.hashing:
+            if blockCheckCount == blockCheck:
+                if self.reporting:
+                    logger.debug('Refreshing Bitcoin block')
+                block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
+                blockCheckCount = 0
+            blockCheckCount += 1
+            hbCount += 1
+            token = nacl.hash.blake2b(nacl.utils.random() + block.encode()).decode()
+            if self.mainHash[0:self.difficulty] == token[0:self.difficulty]:
+                self.hashing = False
+                iFound = True
+                break
+        if iFound:
+            endTime = math.floor(time.time())
+            if self.reporting:
+                logger.info('Found token ' + token)
+                logger.info('took ' + str(endTime - startTime))
+            self.result = token
+    
+    def __init__(self, difficulty, bitcoinNode):
+        self.foundHash = False
+        self.difficulty = difficulty
+
+        logger.debug('Computing difficulty of ' + str(self.difficulty))
+
+        self.mainHash = nacl.hash.blake2b(nacl.utils.random()).decode()
+        self.puzzle = self.mainHash[0:self.difficulty]
+        self.bitcoinNode = bitcoinNode
+        logger.debug('trying to find ' + str(self.mainHash))
+        tOne = threading.Thread(name='one', target=self.pow, args=(True,))
+        tTwo = threading.Thread(name='two', target=self.pow)
+        tThree = threading.Thread(name='three', target=self.pow)
+        tOne.start()
+        tTwo.start()
+        tThree.start()
+        return
+
+    def shutdown(self):
+        self.hashing = False
+        self.puzzle = ''
+    
+    def changeDifficulty(self, newDiff):
+        self.difficulty = newDiff
+
+    def getResult(self):
+        '''Returns the result then sets to false, useful to automatically clear the result'''
+        try:
+            retVal = self.result
+        except AttributeError:
+            retVal = False
+        self.result = False
+        return retVal

onionr/onionrutils.py

@@ -18,29 +18,78 @@
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
 # Misc functions that do not fit in the main api, but are useful
-import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger
+import getpass, sys, requests, os, socket, hashlib, logger, sqlite3, config
+import nacl.signing, nacl.encoding
+
 if sys.version_info < (3, 6):
     try:
         import sha3
     except ModuleNotFoundError:
         logger.fatal('On Python 3 versions prior to 3.6.x, you need the sha3 module')
         sys.exit(1)
+
 class OnionrUtils:
-    '''Various useful functions'''
+    '''
+        Various useful function
+    '''
     def __init__(self, coreInstance):
         self.fingerprintFile = 'data/own-fingerprint.txt'
         self._core = coreInstance
         return
+
+    def sendPM(self, user, message):
+        '''High level function to encrypt a message to a peer and insert it as a block'''
+        return
+    
+    def incrementAddressSuccess(self, address):
+        '''Increase the recorded sucesses for an address'''
+        increment = self._core.getAddressInfo(address, 'success') + 1
+        self._core.setAddressInfo(address, 'success', increment)
+        return
+        
+    def decrementAddressSuccess(self, address):
+        '''Decrease the recorded sucesses for an address'''
+        increment = self._core.getAddressInfo(address, 'success') - 1
+        self._core.setAddressInfo(address, 'success', increment)
+        return
+
+    def mergeKeys(self, newKeyList):
+        '''Merge ed25519 key list to our database'''
+        retVal = False
+        if newKeyList != False:
+            for key in newKeyList:
+                if not key in self._core.listPeers(randomOrder=False):
+                    if self._core.addPeer(key):
+                        retVal = True
+        return retVal
+
+    
+    def mergeAdders(self, newAdderList):
+        '''Merge peer adders list to our database'''
+        retVal = False
+        if newAdderList != False:
+            for adder in newAdderList:
+                if not adder in self._core.listAdders(randomOrder=False):
+                    if self._core.addAddress(adder):
+                        retVal = True
+        return retVal
+
     def localCommand(self, command):
-        '''Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.'''
+        '''
-        config = configparser.ConfigParser()
+            Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.
-        if os.path.exists('data/config.ini'):
+        '''
-            config.read('data/config.ini')
+
-        else:
+        config.reload()
-            return
+
-        requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config['CLIENT']['PORT']) + '/client/?action=' + command + '&token=' + config['CLIENT']['CLIENT HMAC'])
+        # TODO: URL encode parameters, just as an extra measure. May not be needed, but should be added regardless.
+        requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config.get('client')['port']) + '/client/?action=' + command + '&token=' + str(config.get('client')['client_hmac']))
+
+        return
+
     def getPassword(self, message='Enter password: ', confirm = True):
-        '''Get a password without showing the users typing and confirm the input'''
+        '''
+            Get a password without showing the users typing and confirm the input
+        '''
         # Get a password safely with confirmation and return it
         while True:
             print(message)
@@ -50,14 +99,18 @@ class OnionrUtils:
                 pass2 = getpass.getpass()
                 if pass1 != pass2:
                     logger.error("Passwords do not match.")
-                    input()
+                    logger.readline()
                 else:
                     break
             else:
                 break
+
         return pass1
-    def checkPort(self, port, host = ''):
+
-        '''Checks if a port is available, returns bool'''
+    def checkPort(self, port, host=''):
+        '''
+            Checks if a port is available, returns bool
+        '''
         # inspired by https://www.reddit.com/r/learnpython/comments/2i4qrj/how_to_write_a_python_script_that_checks_to_see/ckzarux/
         sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         retVal = False
@@ -68,37 +121,58 @@ class OnionrUtils:
                 retVal = True
         finally:
             sock.close()
+
         return retVal
+
     def checkIsIP(self, ip):
-        '''Check if a string is a valid ipv4 address'''
+        '''
+            Check if a string is a valid IPv4 address
+        '''
         try:
             socket.inet_aton(ip)
         except:
             return False
         else:
             return True
-    def exportMyPubkey(self):
-        '''Export our PGP key if it exists'''
-        if not os.path.exists(self.fingerprintFile):
-            raise Exception("No fingerprint found, cannot export our PGP key.")
-        gpg = gnupg.GPG(homedir='./data/pgp/')
-        with open(self.fingerprintFile,'r') as f:
-            fingerprint = f.read()
-        ascii_armored_public_keys = gpg.export_keys(fingerprint)
-        return ascii_armored_public_keys
 
     def getBlockDBHash(self):
-        '''Return a sha3_256 hash of the blocks DB'''
+        '''
+            Return a sha3_256 hash of the blocks DB
+        '''
         with open(self._core.blockDB, 'rb') as data:
             data = data.read()
         hasher = hashlib.sha3_256()
         hasher.update(data)
         dataHash = hasher.hexdigest()
+
         return dataHash
 
+    def hasBlock(self, hash):
+        '''
+            Check for new block in the list
+        '''
+        conn = sqlite3.connect(self._core.blockDB)
+        c = conn.cursor()
+        if not self.validateHash(hash):
+            raise Exception("Invalid hash")
+        for result in c.execute("SELECT COUNT() FROM hashes where hash='" + hash + "'"):
+            if result[0] >= 1:
+                conn.commit()
+                conn.close()
+                return True
+            else:
+                conn.commit()
+                conn.close()
+                return False
+
     def validateHash(self, data, length=64):
-        '''Validate if a string is a valid hex formatted hash'''
+        '''
+            Validate if a string is a valid hex formatted hash
+        '''
         retVal = True
+        if data == False or data == True:
+            return False
+        data = data.strip()
         if len(data) != length:
             retVal = False
         else:
@@ -106,9 +180,25 @@ class OnionrUtils:
                 int(data, 16)
             except ValueError:
                 retVal = False
+
         return retVal
+
+    def validatePubKey(self, key):
+        '''Validate if a string is a valid base32 encoded Ed25519 key'''
+        retVal = False
+        try:
+            nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
+        except nacl.exceptions.ValueError:
+            pass
+        else:
+            retVal = True
+        return retVal
+
+
     def validateID(self, id):
-        '''validate if a user ID is a valid tor or i2p hidden service'''
+        '''
+            Validate if an address is a valid tor or i2p hidden service
+        '''
         idLength = len(id)
         retVal = True
         idNoDomain = ''
@@ -146,5 +236,5 @@ class OnionrUtils:
                     retVal = False
             if not idNoDomain.isalnum():
                 retVal = False
-        return retVal
 
+        return retVal

onionr/tests.py

@@ -14,7 +14,7 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger
+import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger, btc
 
 class OnionrTests(unittest.TestCase):
     def testPython3(self):
@@ -23,18 +23,21 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(False)
         else:
             self.assertTrue(True)
+
     def testNone(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running simple program run test...')
-        # Test just running ./onionr with no arguments
+
-        blank = os.system('./onionr.py')
+        blank = os.system('./onionr.py --version')
         if blank != 0:
             self.assertTrue(False)
         else:
             self.assertTrue(True)
+
     def testPeer_a_DBCreation(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running peer db creation test...')
+
         if os.path.exists('data/peers.db'):
             os.remove('data/peers.db')
         import core
@@ -44,22 +47,27 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testPeer_b_addPeerToDB(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running peer db insertion test...')
+
         import core
         myCore = core.Core()
         if not os.path.exists('data/peers.db'):
             myCore.createPeerDB()
-        if myCore.addPeer('facebookcorewwwi.onion') and not myCore.addPeer('invalidpeer.onion'):
+        if myCore.addPeer('6M5MXL237OK57ITHVYN5WGHANPGOMKS5C3PJLHBBNKFFJQOIDOJA====') and not myCore.addPeer('NFXHMYLMNFSAU==='):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testData_b_Encrypt(self):
         self.assertTrue(True)
         return
-        logger.debug('--------------------------')
+
+        logger.debug('-'*26 + '\n')
         logger.info('Running data dir encrypt test...')
+
         import core
         myCore = core.Core()
         myCore.dataDirEncrypt('password')
@@ -67,11 +75,14 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
+
     def testData_a_Decrypt(self):
         self.assertTrue(True)
         return
-        logger.debug('--------------------------')
+
+        logger.debug('-'*26 + '\n')
         logger.info('Running data dir decrypt test...')
+
         import core
         myCore = core.Core()
         myCore.dataDirDecrypt('password')
@@ -79,34 +90,82 @@ class OnionrTests(unittest.TestCase):
             self.assertTrue(True)
         else:
             self.assertTrue(False)
-    def testPGPGen(self):
+
-        logger.debug('--------------------------')
+    def testConfig(self):
-        logger.info('Running PGP key generation test...')
+        logger.debug('-'*26 + '\n')
-        if os.path.exists('data/pgp/'):
+        logger.info('Running simple configuration test...')
-            self.assertTrue(True)
+
-        else:
+        import config
-            import core, netcontroller
+
-            myCore = core.Core()
+        config.check()
-            net = netcontroller.NetController(1337)
+        config.reload()
-            net.startTor()
+        configdata = str(config.get_config())
-            torID = open('data/hs/hostname').read()
+
-            myCore.generateMainPGP(torID)
+        config.set('testval', 1337)
-            if os.path.exists('data/pgp/'):
+        if not config.get('testval', None) is 1337:
-                self.assertTrue(True)
-    def testHMACGen(self):
-        logger.debug('--------------------------')
-        logger.info('Running HMAC generation test...')
-        # Test if hmac key generation is working
-        import core
-        myCore = core.Core()
-        key = myCore.generateHMAC()
-        if len(key) > 10:
-            self.assertTrue(True)
-        else:
             self.assertTrue(False)
+
+        config.set('testval')
+        if not config.get('testval', None) is None:
+            self.assertTrue(False)
+
+        config.save()
+        config.reload()
+
+        if not str(config.get_config()) == configdata:
+            self.assertTrue(False)
+
+        self.assertTrue(True)
+
+    def testBitcoinNode(self):
+        # temporarily disabled- this takes a lot of time the CI doesn't have
+        self.assertTrue(True)
+        #logger.debug('-'*26 + '\n')
+        #logger.info('Running bitcoin node test...')
+
+        #sbitcoin = btc.OnionrBTC()
+
+    def testPluginReload(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running simple plugin reload test...')
+
+        import onionrplugins
+        try:
+            onionrplugins.reload('test')
+            self.assertTrue(True)
+        except:
+            self.assertTrue(False)
+
+    def testPluginStopStart(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running simple plugin restart test...')
+
+        import onionrplugins
+        try:
+            onionrplugins.start('test')
+            onionrplugins.stop('test')
+            self.assertTrue(True)
+        except:
+            self.assertTrue(False)
+
+    def testPluginEvent(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running plugin event test...')
+
+        import onionrplugins as plugins, onionrevents as events
+
+        plugins.start('test')
+        if not events.call(plugins.get_plugin('test'), 'test'):
+            self.assertTrue(False)
+
+        events.event('test', data = {'tests': self})
+
+        self.assertTrue(True)
+
     def testQueue(self):
-        logger.debug('--------------------------')
+        logger.debug('-'*26 + '\n')
         logger.info('Running daemon queue test...')
+
         # test if the daemon queue can read/write data
         import core
         myCore = core.Core()
@@ -124,4 +183,32 @@ class OnionrTests(unittest.TestCase):
         if command[0] == 'testCommand':
             if myCore.daemonQueue() == False:
                 logger.info('Succesfully added and read command')
+
+    def testHashValidation(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running hash validation test...')
+
+        import core
+        myCore = core.Core()
+        if not myCore._utils.validateHash("$324dfgfdg") and myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2") and not myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd$"):
+            self.assertTrue(True)
+        else:
+            self.assertTrue(False)
+
+    def testAddAdder(self):
+        logger.debug('-'*26 + '\n')
+        logger.info('Running address add+remove test')
+
+        import core
+        myCore = core.Core()
+        if not os.path.exists('data/address.db'):
+            myCore.createAddressDB()
+        if myCore.addAddress('facebookcorewwwi.onion') and not myCore.removeAddress('invalid'):
+            if myCore.removeAddress('facebookcorewwwi.onion'):
+                self.assertTrue(True)
+            else:
+                self.assertTrue(False)
+        else:
+            self.assertTrue(False)
+
 unittest.main()

onionr/timedHmac.py

@@ -16,12 +16,12 @@ import hmac, base64, time, math
 class TimedHMAC:
     def __init__(self, base64Key, data, hashAlgo):
         '''
-        base64Key = base64 encoded key
+            base64Key = base64 encoded key
-        data = data to hash
+            data = data to hash
-        expire = time expiry in epoch
+            expire = time expiry in epoch
-        hashAlgo = string in hashlib.algorithms_available
+            hashAlgo = string in hashlib.algorithms_available
 
-        Maximum of 10 seconds grace period
+            Maximum of 10 seconds grace period
         '''
         self.data = data
         self.expire = math.floor(time.time())
@@ -30,6 +30,7 @@ class TimedHMAC:
         generatedHMAC = hmac.HMAC(base64.b64decode(base64Key).decode(), digestmod=self.hashAlgo)
         generatedHMAC.update(data + expire)
         self.HMACResult = generatedHMAC.hexdigest()
+
         return
 
     def check(self, data):

readme.md

@@ -1,18 +1,38 @@
-# Onionr
+![Onionr logo](./docs/onionr-logo.png)
 
 [![Build Status](https://travis-ci.org/beardog108/onionr.svg?branch=master)](https://travis-ci.org/beardog108/onionr)
+[![Open Source Love](https://badges.frapsoft.com/os/v3/open-source.png?v=103)](https://github.com/ellerbrock/open-source-badges/)
 
-P2P microblogging platform and social network, using Tor & I2P.
+
+Anonymous P2P platform, using Tor & I2P.
 
 Major work in progress.
 
 ***THIS SOFTWARE IS NOT USABLE OR SAFE YET.***
 
 
+**Roadmap/features:**
+
+* [X] Fully p2p/decentralized, no trackers or other single points of failure
+* [X] High level of anonymity
+* [ ] End to end encryption where applicable
+* [X] Optional non-encrypted blocks, useful for blog posts or public file sharing
+* [ ] Easy API system for integration to websites
+
 # Development
 
 This software is in heavy development. If for some reason you want to get involved, get in touch first.
 
+**Onionr API and functionality is subject to non-backwards compatible change during development**
+
+# Donate
+
+Bitcoin/Bitcoin Cash: 1onion55FXzm6h8KQw3zFw2igpHcV7LPq
+
 ## Disclaimer
 
 The Tor Project, I2P developers, and anyone else do not own, create, or endorse this project, and are not otherwise involved.
+
+The badges (besides travis-ci build) are by Maik Ellerbrock is licensed under a Creative Commons Attribution 4.0 International License.
+
+The onion in the Onionr logo is adapted from [this](https://commons.wikimedia.org/wiki/File:Red_Onion_on_White.JPG) image by Colin on Wikimedia under a Creative Commons Attribution-Share Alike 3.0 Unported license

requirements.txt

@@ -1,11 +1,8 @@
 PyNaCl==1.2.1
-gnupg==2.3.1
+requests==2.12.4
 Flask==0.12.2
-requests==2.18.4
-urllib3==1.22
 simple_crypt==4.1.7
+urllib3==1.19.1
 sha3==0.2.1
-pycrypto==2.6.1
-pynacl==1.2.1
 PySocks==1.6.8
 bitpeer.py==0.4.7.5

test.sh

@@ -1,3 +0,0 @@
-#!/bin/sh
-cd onionr
-./tests.py