Merge branch 'crypto' of github.com:beardog108/onionr into crypto

This commit is contained in:
Kevin Froman 2018-02-17 13:04:38 -06:00
commit b6cfe0154d
8 changed files with 36 additions and 154 deletions

View File

@ -1,71 +1,48 @@
# Onionr Protocol Spec # Onionr Protocol Spec v2
A social network/microblogging platform for Tor & I2P A P2P platform for Tor & I2P
Draft Dec 25 2017
# Overview # Overview
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter. Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P). There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
User IDs are simply Tor onion service/I2P host id + PGP fingerprint. User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
Clients consolidate feeds from peers into 1 “timeline” using RSS format. Private blocks are only able to be read by the intended peer.
Private messages are only accessible by the intended peer based on the PGP id. All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
## Goals: ## Goals:
• Selective sharing of information with friends & public • Selective sharing of information
• Secure & semi-anonymous direct messaging • Secure & semi-anonymous direct messaging
• Forward secrecy • Forward secrecy
• Defense in depth • Defense in depth
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”) • Data should be secure for years to come
• Decentralization • Decentralization
* Avoid browser-based exploits that plague similar software * Avoid browser-based exploits that plague similar software
* Avoid timing attacks & unexpected metadata leaks * Avoid timing attacks & unexpected metadata leaks
## Assumptions:
• Tor & I2Ps transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
• All traffic is logged indefinitely by powerful adversaries
## Protocol ## Protocol
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
## Connections ## Connections
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry. When a node first comes online, it attempts to bootstrap using a default list provided by a client.
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default. When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
Every 100kb or every 2 hours is a recommended default.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends: * Friends:
* Encrypted friends only posts to one another * Encrypted friends only posts to one another
* Usually less strict rate & storage limits * Usually less strict rate & storage limits
* OPTIONALLY sign one anothers keys. Users may not want to do this in order to avoid exposing their entire friends list. * Strangers:
• Strangers:
* Used for storage of encrypted or public information * Used for storage of encrypted or public information
* Can only read public posts * Can only read public posts
* Usually stricter rate & storage limits * Usually stricter rate & storage limits
## Data Storage/Delivery
Posts (public or friends only) are stored across the network.
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
Data SHOULD be stored as long as the nodes user prefers and only erased once disk quota is reached due to new data.
Posts
Posts can contain text and images. All posts MUST be time stamped.
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
All received posts must be verified to be stored and/or displayed to the user.
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
Posts have two settings:
• Friends only:
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
• Public:
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
## Private Messages
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
When both peers are online messages SHOULD be dispatched directly between peers.
All messages must be verified prior to being displayed.
Clients SHOULD allow configurable message padding.
## Spam mitigation ## Spam mitigation
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work). To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).

View File

@ -20,7 +20,7 @@
import flask import flask
from flask import request, Response, abort from flask import request, Response, abort
from multiprocessing import Process from multiprocessing import Process
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
from core import Core from core import Core
import onionrutils, onionrcrypto import onionrutils, onionrcrypto
@ -140,8 +140,6 @@ class API:
resp = Response(self._utils.getBlockDBHash()) resp = Response(self._utils.getBlockDBHash())
elif action == 'getBlockHashes': elif action == 'getBlockHashes':
resp = Response(self._core.getBlockList()) resp = Response(self._core.getBlockList())
elif action == 'getPGP':
resp = Response(self._utils.exportMyPubkey())
# setData should be something the communicator initiates, not this api # setData should be something the communicator initiates, not this api
elif action == 'getData': elif action == 'getData':
resp = self._core.getData(data) resp = self._core.getData(data)

View File

@ -40,13 +40,6 @@ class OnionrCommunicate:
self.peerData = {} # Session data for peers (recent reachability, speed, etc) self.peerData = {} # Session data for peers (recent reachability, speed, etc)
# get our own PGP fingerprint
fingerprintFile = 'data/own-fingerprint.txt'
if not os.path.exists(fingerprintFile):
self._core.generateMainPGP(torID)
with open(fingerprintFile,'r') as f:
self.pgpOwnFingerprint = f.read()
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
if os.path.exists(self._core.queueDB): if os.path.exists(self._core.queueDB):
self._core.clearDaemonQueue() self._core.clearDaemonQueue()
while True: while True:

View File

@ -17,7 +17,7 @@
You should have received a copy of the GNU General Public License You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. along with this program. If not, see <https://www.gnu.org/licenses/>.
''' '''
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
#from Crypto.Cipher import AES #from Crypto.Cipher import AES
#from Crypto import Random #from Crypto import Random
import netcontroller import netcontroller
@ -38,10 +38,8 @@ class Core:
''' '''
self.queueDB = 'data/queue.db' self.queueDB = 'data/queue.db'
self.peerDB = 'data/peers.db' self.peerDB = 'data/peers.db'
self.ownPGPID = ''
self.blockDB = 'data/blocks.db' self.blockDB = 'data/blocks.db'
self.blockDataLocation = 'data/blocks/' self.blockDataLocation = 'data/blocks/'
self.gpgHome = './data/pgp/'
self._utils = onionrutils.OnionrUtils(self) self._utils = onionrutils.OnionrUtils(self)
self._crypto = onionrcrypto.OnionrCrypto(self) self._crypto = onionrcrypto.OnionrCrypto(self)
@ -55,28 +53,6 @@ class Core:
return return
def generateMainPGP(self, myID):
'''
Generate the main PGP key for our client. Should not be done often.
Uses own PGP home folder in the data/ directory
'''
gpg = gnupg.GPG(homedir=self.gpgHome)
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
key = gpg.gen_key(input_data)
logger.info("Generating PGP key, this will take some time..")
while key.status != "key created":
time.sleep(0.5)
print(key.status)
logger.info("Finished generating PGP key")
# Write the key
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
myFingerpintFile.write(key.fingerprint)
myFingerpintFile.close()
return
def addPeer(self, peerID, name=''): def addPeer(self, peerID, name=''):
''' '''
Add a peer by their ID, with an optional name, to the peer database Add a peer by their ID, with an optional name, to the peer database
@ -104,8 +80,7 @@ class Core:
c.execute('''CREATE TABLE peers( c.execute('''CREATE TABLE peers(
ID text not null, ID text not null,
name text, name text,
pgpKey text, pubkey text,
hmacKey text,
blockDBHash text, blockDBHash text,
forwardKey text, forwardKey text,
dateSeen not null, dateSeen not null,
@ -335,7 +310,6 @@ class Core:
id text 0 id text 0
name text, 1 name text, 1
pgpKey text, 2
hmacKey text, 3 hmacKey text, 3
blockDBHash text, 4 blockDBHash text, 4
forwardKey text, 5 forwardKey text, 5
@ -346,7 +320,7 @@ class Core:
conn = sqlite3.connect(self.peerDB) conn = sqlite3.connect(self.peerDB)
c = conn.cursor() c = conn.cursor()
command = (peer,) command = (peer,)
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8} infoNumbers = {'id': 0, 'name': 1, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
info = infoNumbers[info] info = infoNumbers[info]
iterCount = 0 iterCount = 0
retVal = '' retVal = ''
@ -369,7 +343,7 @@ class Core:
c = conn.cursor() c = conn.cursor()
command = (data, peer) command = (data, peer)
# TODO: validate key on whitelist # TODO: validate key on whitelist
if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'): if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
raise Exception("Got invalid database key when setting peer info") raise Exception("Got invalid database key when setting peer info")
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command) c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
conn.commit() conn.commit()

View File

@ -17,7 +17,7 @@
You should have received a copy of the GNU General Public License You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>. along with this program. If not, see <https://www.gnu.org/licenses/>.
''' '''
import nacl, gnupg import nacl
class OnionrCrypto: class OnionrCrypto:
def __init__(self, coreInstance): def __init__(self, coreInstance):
@ -30,14 +30,8 @@ class OnionrCrypto:
def symmetricPeerDecrypt(self, data, key): def symmetricPeerDecrypt(self, data, key):
return return
def rsaEncrypt(self, peer, data):
return
def verifyPGP(self, peer, signature):
'''Verify PGP signed data'''
gpg = gnupg.GPG(homedir=self._core.gpgHome)
def generateSymmetric(): def generateSymmetric():
return return
def generateHMAC(): def generateHMAC():
return return

View File

@ -18,7 +18,7 @@
along with this program. If not, see <https://www.gnu.org/licenses/>. along with this program. If not, see <https://www.gnu.org/licenses/>.
''' '''
# Misc functions that do not fit in the main api, but are useful # Misc functions that do not fit in the main api, but are useful
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3 import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
if sys.version_info < (3, 6): if sys.version_info < (3, 6):
try: try:
import sha3 import sha3
@ -93,19 +93,6 @@ class OnionrUtils:
else: else:
return True return True
def exportMyPubkey(self):
'''
Export our PGP key if it exists
'''
if not os.path.exists(self.fingerprintFile):
raise Exception("No fingerprint found, cannot export our PGP key.")
gpg = gnupg.GPG(homedir='./data/pgp/')
with open(self.fingerprintFile,'r') as f:
fingerprint = f.read()
ascii_armored_public_keys = gpg.export_keys(fingerprint)
return ascii_armored_public_keys
def getBlockDBHash(self): def getBlockDBHash(self):
''' '''
Return a sha3_256 hash of the blocks DB Return a sha3_256 hash of the blocks DB
@ -154,17 +141,6 @@ class OnionrUtils:
return retVal return retVal
def getPeerPGPFingerprint(self, peer):
'''
Get peer's PGP fingerprint
'''
retData = ''
gpg = gnupg.GPG(homedir=self._core.gpgHome)
for i in gpg.list_keys():
if peer in i['uids'][0]:
retData = i['fingerprint']
return retData
def validateID(self, id): def validateID(self, id):
''' '''
Validate if a user ID is a valid tor or i2p hidden service Validate if a user ID is a valid tor or i2p hidden service

View File

@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
else: else:
self.assertTrue(False) self.assertTrue(False)
def testPGPGen(self):
logger.debug('--------------------------')
logger.info('Running PGP key generation test...')
if os.path.exists('data/pgp/'):
self.assertTrue(True)
else:
import core, netcontroller
myCore = core.Core()
net = netcontroller.NetController(1337)
net.startTor()
torID = open('data/hs/hostname').read()
myCore.generateMainPGP(torID)
if os.path.exists('data/pgp/'):
self.assertTrue(True)
def testHMACGen(self):
logger.debug('--------------------------')
logger.info('Running HMAC generation test...')
# Test if hmac key generation is working
import core
myCore = core.Core()
key = myCore.generateHMAC()
if len(key) > 10:
self.assertTrue(True)
else:
self.assertTrue(False)
def testQueue(self): def testQueue(self):
logger.debug('--------------------------') logger.debug('--------------------------')
logger.info('Running daemon queue test...') logger.info('Running daemon queue test...')

View File

@ -1,10 +1,7 @@
PyNaCl==1.2.1 PyNaCl==1.2.1
gnupg==2.3.1 requests==2.12.4
Flask==0.12.2 Flask==0.12.2
requests==2.18.4
urllib3==1.22
simple_crypt==4.1.7 simple_crypt==4.1.7
urllib3==1.19.1
sha3==0.2.1 sha3==0.2.1
pycrypto==2.6.1
pynacl==1.2.1
PySocks==1.6.8 PySocks==1.6.8