Merge branch 'crypto' of github.com:beardog108/onionr into crypto

docs/onionr-draft.md

@@ -1,71 +1,48 @@
-# Onionr Protocol Spec
+# Onionr Protocol Spec v2
 
-A social network/microblogging platform for Tor & I2P
+A P2P platform for Tor & I2P
-
-Draft Dec 25 2017
 
 # Overview
 
 Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
 There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
-User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
+User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
-Clients consolidate feeds from peers into 1 “timeline” using RSS format.
+Private blocks are only able to be read by the intended peer.
-Private messages are only accessible by the intended peer based on the PGP id.
+All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
-Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
+
-All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
 ## Goals:
-    • Selective sharing of information with friends & public
+    • Selective sharing of information
     • Secure & semi-anonymous direct messaging
     • Forward secrecy
     • Defense in depth
-    • Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
+    • Data should be secure for years to come
     • Decentralization
     * Avoid browser-based exploits that plague similar software
     * Avoid timing attacks & unexpected metadata leaks
-## Assumptions:
+
-    • Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
-    • All traffic is logged indefinitely by powerful adversaries
 ## Protocol
-Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
+
+Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
+
 ## Connections
-    When a node first comes online, it attempts to bootstrap using a default list provided by a client.
+
-    When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
+When a node first comes online, it attempts to bootstrap using a default list provided by a client.
-    HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
+When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
-    All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
+
-    Peer Types
+Salsa20 keys are regenerated either every X many communications with a peer or every X minutes. 
+
+Every 100kb or every 2 hours is a recommended default.
+
+All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
+Peer Types
     * Friends:
         * Encrypted ‘friends only’ posts to one another
         * Usually less strict rate & storage limits
-            * OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
+    * Strangers:
-        • Strangers:
         * Used for storage of encrypted or public information
         * Can only read public posts
         * Usually stricter rate & storage limits
-## Data Storage/Delivery
 
-    Posts (public or friends only) are stored across the network.
-    Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
-    Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
-    Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
-    Posts
-    Posts can contain text and images. All posts MUST be time stamped.
-    Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
-    All received posts must be verified to be stored and/or displayed to the user.
-
-    All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
-
-     Posts have two settings:
-        • Friends only:
-            ◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
-        • Public:
-            ◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
-## Private Messages
-
-    Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
-    When both peers are online messages SHOULD be dispatched directly between peers.
-    All messages must be verified prior to being displayed.
-
-    Clients SHOULD allow configurable message padding.
 ## Spam mitigation
 
 To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).

onionr/api.py

@@ -20,7 +20,7 @@
 import flask
 from flask import request, Response, abort
 from multiprocessing import Process
-import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
+import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
 
 from core import Core
 import onionrutils, onionrcrypto
@@ -140,8 +140,6 @@ class API:
                 resp = Response(self._utils.getBlockDBHash())
             elif action == 'getBlockHashes':
                 resp = Response(self._core.getBlockList())
-            elif action == 'getPGP':
-                resp = Response(self._utils.exportMyPubkey())
             # setData should be something the communicator initiates, not this api
             elif action == 'getData':
                 resp = self._core.getData(data)

onionr/communicator.py

@@ -40,13 +40,6 @@ class OnionrCommunicate:
 
         self.peerData = {} # Session data for peers (recent reachability, speed, etc)
 
-        # get our own PGP fingerprint
-        fingerprintFile = 'data/own-fingerprint.txt'
-        if not os.path.exists(fingerprintFile):
-            self._core.generateMainPGP(torID)
-        with open(fingerprintFile,'r') as f:
-            self.pgpOwnFingerprint = f.read()
-        logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
         if os.path.exists(self._core.queueDB):
             self._core.clearDaemonQueue()
         while True:

onionr/core.py

@@ -17,7 +17,7 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
+import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
 #from Crypto.Cipher import AES
 #from Crypto import Random
 import netcontroller
@@ -38,10 +38,8 @@ class Core:
         '''
         self.queueDB = 'data/queue.db'
         self.peerDB = 'data/peers.db'
-        self.ownPGPID = ''
         self.blockDB = 'data/blocks.db'
         self.blockDataLocation = 'data/blocks/'
-        self.gpgHome = './data/pgp/'
         self._utils = onionrutils.OnionrUtils(self)
         self._crypto = onionrcrypto.OnionrCrypto(self)
 
@@ -55,28 +53,6 @@ class Core:
 
         return
 
-    def generateMainPGP(self, myID):
-        '''
-            Generate the main PGP key for our client. Should not be done often.
-
-            Uses own PGP home folder in the data/ directory
-        '''
-        gpg = gnupg.GPG(homedir=self.gpgHome)
-        input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
-        key = gpg.gen_key(input_data)
-        logger.info("Generating PGP key, this will take some time..")
-        while key.status != "key created":
-            time.sleep(0.5)
-            print(key.status)
-
-        logger.info("Finished generating PGP key")
-        # Write the key
-        myFingerpintFile = open('data/own-fingerprint.txt', 'w')
-        myFingerpintFile.write(key.fingerprint)
-        myFingerpintFile.close()
-
-        return
-
     def addPeer(self, peerID, name=''):
         '''
             Add a peer by their ID, with an optional name, to the peer database
@@ -104,8 +80,7 @@ class Core:
         c.execute('''CREATE TABLE peers(
             ID text not null,
             name text,
-            pgpKey text,
+            pubkey text,
-            hmacKey text,
             blockDBHash text,
             forwardKey text,
             dateSeen not null,
@@ -335,7 +310,6 @@ class Core:
 
             id text             0
             name text,          1
-            pgpKey text,        2
             hmacKey text,       3
             blockDBHash text,   4
             forwardKey text,    5
@@ -346,7 +320,7 @@ class Core:
         conn = sqlite3.connect(self.peerDB)
         c = conn.cursor()
         command = (peer,)
-        infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
+        infoNumbers = {'id': 0, 'name': 1, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
         info = infoNumbers[info]
         iterCount = 0
         retVal = ''
@@ -369,7 +343,7 @@ class Core:
         c = conn.cursor()
         command = (data, peer)
         # TODO: validate key on whitelist
-        if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
+        if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
             raise Exception("Got invalid database key when setting peer info")
         c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
         conn.commit()

onionr/onionrcrypto.py

@@ -17,7 +17,7 @@
     You should have received a copy of the GNU General Public License
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
-import nacl, gnupg
+import nacl
 
 class OnionrCrypto:
     def __init__(self, coreInstance):
@@ -30,14 +30,8 @@ class OnionrCrypto:
     def symmetricPeerDecrypt(self, data, key):
         return
     
-    def rsaEncrypt(self, peer, data):
-        return
-    
-    def verifyPGP(self, peer, signature):
-        '''Verify PGP signed data'''
-        gpg = gnupg.GPG(homedir=self._core.gpgHome)
-    
     def generateSymmetric():
         return
+
     def generateHMAC():
         return

onionr/onionrutils.py

@@ -18,7 +18,7 @@
     along with this program.  If not, see <https://www.gnu.org/licenses/>.
 '''
 # Misc functions that do not fit in the main api, but are useful
-import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3
+import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
 if sys.version_info < (3, 6):
     try:
         import sha3
@@ -93,19 +93,6 @@ class OnionrUtils:
         else:
             return True
 
-    def exportMyPubkey(self):
-        '''
-            Export our PGP key if it exists
-        '''
-        if not os.path.exists(self.fingerprintFile):
-            raise Exception("No fingerprint found, cannot export our PGP key.")
-        gpg = gnupg.GPG(homedir='./data/pgp/')
-        with open(self.fingerprintFile,'r') as f:
-            fingerprint = f.read()
-        ascii_armored_public_keys = gpg.export_keys(fingerprint)
-
-        return ascii_armored_public_keys
-
     def getBlockDBHash(self):
         '''
             Return a sha3_256 hash of the blocks DB
@@ -154,17 +141,6 @@ class OnionrUtils:
 
         return retVal
 
-    def getPeerPGPFingerprint(self, peer):
-        '''
-            Get peer's PGP fingerprint
-        '''
-        retData = ''
-        gpg = gnupg.GPG(homedir=self._core.gpgHome)
-        for i in gpg.list_keys():
-            if peer in i['uids'][0]:
-                retData = i['fingerprint']
-        return retData
-
     def validateID(self, id):
         '''
             Validate if a user ID is a valid tor or i2p hidden service

onionr/tests.py

@@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
         else:
             self.assertTrue(False)
 
-    def testPGPGen(self):
-        logger.debug('--------------------------')
-        logger.info('Running PGP key generation test...')
-        if os.path.exists('data/pgp/'):
-            self.assertTrue(True)
-        else:
-            import core, netcontroller
-            myCore = core.Core()
-            net = netcontroller.NetController(1337)
-            net.startTor()
-            torID = open('data/hs/hostname').read()
-            myCore.generateMainPGP(torID)
-            if os.path.exists('data/pgp/'):
-                self.assertTrue(True)
-
-    def testHMACGen(self):
-        logger.debug('--------------------------')
-        logger.info('Running HMAC generation test...')
-        # Test if hmac key generation is working
-        import core
-        myCore = core.Core()
-        key = myCore.generateHMAC()
-        if len(key) > 10:
-            self.assertTrue(True)
-        else:
-            self.assertTrue(False)
-
     def testQueue(self):
         logger.debug('--------------------------')
         logger.info('Running daemon queue test...')

requirements.txt

@@ -1,10 +1,7 @@
 PyNaCl==1.2.1
-gnupg==2.3.1
+requests==2.12.4
 Flask==0.12.2
-requests==2.18.4
-urllib3==1.22
 simple_crypt==4.1.7
+urllib3==1.19.1
 sha3==0.2.1
-pycrypto==2.6.1
-pynacl==1.2.1
 PySocks==1.6.8