kev/Onionr
1
0
Fork 0
Code Issues 24 Pull Requests Projects Releases 2 Wiki Activity

Merge branch 'crypto' of https://github.com/beardog108/onionr into crypto

Browse Source
This commit is contained in:
Arinerron 2018-02-21 22:42:04 -08:00
commit 005273b52c
14 changed files with 196 additions and 213 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -6,3 +6,4 @@ onionr/*.pyc
onionr/*.log
onionr/data/hs/hostname
onionr/data/*
onionr/gnupg/*

12
ISSUE_TEMPLATE.md Normal file
View File

@ -0,0 +1,12 @@
# Expected Behavior
# Actual Behavior
# Steps to Reproduce
# Version Information
Onionr:
OS:
Python:
Tor:
I2P:

74
docs/onionr-draft.md
View File

@ -1,74 +1,50 @@
# Onionr Protocol Spec
# Onionr Protocol Spec v2
A social network/microblogging platform for Tor & I2P
Draft Dec 25 2017
notes:
Use Blowfish in addition with AES?
A P2P platform for Tor & I2P
# Overview
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
Clients consolidate feeds from peers into 1 “timeline” using RSS format.
Private messages are only accessible by the intended peer based on the PGP id.
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
Private blocks are only able to be read by the intended peer.
All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
## Goals:
• Selective sharing of information with friends & public
• Selective sharing of information
• Secure & semi-anonymous direct messaging
• Forward secrecy
• Defense in depth
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
• Data should be secure for years to come
• Decentralization
* Avoid browser-based exploits that plague similar software
* Avoid timing attacks & unexpected metadata leaks
## Assumptions:
• Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
• All traffic is logged indefinitely by powerful adversaries
## Protocol
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
Blocks may be encrypted using Curve25519.
## Connections
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10 communications or every 24 hours is a recommended default.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
Every 100kb or every 2 hours is a recommended default.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends:
* Encrypted ‘friends only’ posts to one another
* Usually less strict rate & storage limits
* OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
• Strangers:
* Strangers:
* Used for storage of encrypted or public information
* Can only read public posts
* Usually stricter rate & storage limits
## Data Storage/Delivery
Posts (public or friends only) are stored across the network.
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
Posts
Posts can contain text and images. All posts MUST be time stamped.
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
All received posts must be verified to be stored and/or displayed to the user.
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
Posts have two settings:
• Friends only:
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
• Public:
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
## Private Messages
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
When both peers are online messages SHOULD be dispatched directly between peers.
All messages must be verified prior to being displayed.
Clients SHOULD allow configurable message padding.
## Spam mitigation
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).

17
onionr/api.py
View File

@ -20,10 +20,10 @@
import flask
from flask import request, Response, abort
from multiprocessing import Process
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, os, logger
from core import Core
import onionrutils
import onionrutils, onionrcrypto
class API:
'''
Main HTTP API (Flask)
@ -47,7 +47,7 @@ class API:
if os.path.exists('dev-enabled'):
self._developmentMode = True
logger.set_level(logger.LEVEL_DEBUG)
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
#logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
else:
self._developmentMode = False
logger.set_level(logger.LEVEL_INFO)
@ -56,6 +56,7 @@ class API:
self.debug = debug
self._privateDelayTime = 3
self._core = Core()
self._crypto = onionrcrypto.OnionrCrypto(self._core)
self._utils = onionrutils.OnionrUtils(self._core)
app = flask.Flask(__name__)
bindPort = int(self.config['CLIENT']['PORT'])
@ -131,14 +132,14 @@ class API:
pass
elif action == 'ping':
resp = Response("pong!")
elif action == 'setHMAC':
pass
elif action == 'getHMAC':
resp = Response(self._crypto.generateSymmetric())
elif action == 'getSymmetric':
resp = Response(self._crypto.generateSymmetric())
elif action == 'getDBHash':
resp = Response(self._utils.getBlockDBHash())
elif action == 'getBlockHashes':
resp = Response(self._core.getBlockList())
elif action == 'getPGP':
resp = Response(self._utils.exportMyPubkey())
# setData should be something the communicator initiates, not this api
elif action == 'getData':
resp = self._core.getData(data)
@ -171,7 +172,7 @@ class API:
resp = Response("Invalid request")
return resp
if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
logger.debug('Client token: ' + logger.colors.underline + self.clientToken)

43
onionr/communicator.py
View File

@ -20,7 +20,7 @@ and code to operate as a daemon, getting commands from the command queue databas
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sqlite3, requests, hmac, hashlib, time, sys, os, math, logger, urllib.parse
import core, onionrutils
import core, onionrutils, onionrcrypto
class OnionrCommunicate:
def __init__(self, debug, developmentMode):
@ -31,6 +31,7 @@ class OnionrCommunicate:
'''
self._core = core.Core()
self._utils = onionrutils.OnionrUtils(self._core)
self._crypto = onionrcrypto.OnionrCrypto(self._core)
blockProcessTimer = 0
blockProcessAmount = 5
heartBeatTimer = 0
@ -40,13 +41,6 @@ class OnionrCommunicate:
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
# get our own PGP fingerprint
fingerprintFile = 'data/own-fingerprint.txt'
if not os.path.exists(fingerprintFile):
self._core.generateMainPGP(torID)
with open(fingerprintFile,'r') as f:
self.pgpOwnFingerprint = f.read()
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
if os.path.exists(self._core.queueDB):
self._core.clearDaemonQueue()
while True:
@ -70,39 +64,6 @@ class OnionrCommunicate:
return
def getRemotePeerKey(self, peerID):
'''
This function contacts a peer and gets their main PGP key.
This is safe because Tor or I2P is used, but it does not ensure that the person is who they say they are
'''
url = 'http://' + peerID + '/public/?action=getPGP'
r = requests.get(url, headers=headers)
response = r.text
return response
def shareHMAC(self, peerID, key):
'''
This function shares an HMAC key to a peer
'''
return
def getPeerProof(self, peerID):
'''
This function gets the current peer proof requirement
'''
return
def sendPeerProof(self, peerID, data):
'''
This function sends the proof result to a peer previously fetched with getPeerProof
'''
return
def lookupBlocks(self):
'''
Lookup blocks and merge new ones

92
onionr/core.py
View File

@ -17,12 +17,12 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
from Crypto.Cipher import AES
from Crypto import Random
import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
#from Crypto.Cipher import AES
#from Crypto import Random
import netcontroller
import onionrutils
import onionrutils, onionrcrypto
if sys.version_info < (3, 6):
try:
@ -38,40 +38,20 @@ class Core:
'''
self.queueDB = 'data/queue.db'
self.peerDB = 'data/peers.db'
self.ownPGPID = ''
self.blockDB = 'data/blocks.db'
self.blockDataLocation = 'data/blocks/'
self._utils = onionrutils.OnionrUtils(self)
self.addressDB = 'data/address.db'
if not os.path.exists('data/'):
os.mkdir('data/')
if not os.path.exists('data/blocks/'):
os.mkdir('data/blocks/')
if not os.path.exists(self.blockDB):
self.createBlockDB()
return
def generateMainPGP(self, myID):
'''
Generate the main PGP key for our client. Should not be done often.
Uses own PGP home folder in the data/ directory
'''
gpg = gnupg.GPG(homedir='./data/pgp/')
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
key = gpg.gen_key(input_data)
logger.info("Generating PGP key, this will take some time..")
while key.status != "key created":
time.sleep(0.5)
print(key.status)
logger.info("Finished generating PGP key")
# Write the key
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
myFingerpintFile.write(key.fingerprint)
myFingerpintFile.close()
self._utils = onionrutils.OnionrUtils(self)
# Initialize the crypto object
self._crypto = onionrcrypto.OnionrCrypto(self)
return
@ -82,7 +62,7 @@ class Core:
DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion
'''
# This function simply adds a peer to the DB
if not self._utils.validateID(peerID):
if not self._utils.validatePubKey(peerID):
return False
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
@ -92,6 +72,29 @@ class Core:
conn.close()
return True
def createAddressDB(self):
'''
Generate the address database
types:
1: I2P b32 address
2: Tor v2 (like facebookcorewwwi.onion)
3: Tor v3
'''
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
c.execute('''CREATE TABLE adders(
address text,
type int,
knownPeer text,
speed int,
success int,
failure int
);
''')
conn.commit()
conn.close()
def createPeerDB(self):
'''
Generate the peer sqlite3 database and populate it with the peers table.
@ -102,8 +105,7 @@ class Core:
c.execute('''CREATE TABLE peers(
ID text not null,
name text,
pgpKey text,
hmacKey text,
adders text,
blockDBHash text,
forwardKey text,
dateSeen not null,
@ -112,7 +114,6 @@ class Core:
''')
conn.commit()
conn.close()
return
def createBlockDB(self):
@ -300,14 +301,6 @@ class Core:
return
def generateHMAC(self, length=32):
'''
Generate and return an HMAC key
'''
key = base64.b64encode(os.urandom(length))
return key
def listPeers(self, randomOrder=True):
'''
Return a list of peers
@ -322,7 +315,7 @@ class Core:
peers = c.execute('SELECT * FROM peers;')
peerList = []
for i in peers:
peerList.append(i[0])
peerList.append(i[2])
conn.close()
return peerList
@ -333,18 +326,17 @@ class Core:
id text 0
name text, 1
pgpKey text, 2
hmacKey text, 3
blockDBHash text, 4
forwardKey text, 5
dateSeen not null, 7
bytesStored int, 8
trust int 9
adders text, 2
blockDBHash text, 3
forwardKey text, 4
dateSeen not null, 5
bytesStored int, 6
trust int 7
'''
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
command = (peer,)
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
infoNumbers = {'id': 0, 'name': 1, 'adders': 2, 'blockDBHash': 3, 'forwardKey': 4, 'dateSeen': 5, 'bytesStored': 6, 'trust': 7}
info = infoNumbers[info]
iterCount = 0
retVal = ''
@ -367,7 +359,7 @@ class Core:
c = conn.cursor()
command = (data, peer)
# TODO: validate key on whitelist
if key not in ('id', 'text', 'name', 'pgpKey', 'hmacKey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
raise Exception("Got invalid database key when setting peer info")
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
conn.commit()

5
onionr/netcontroller.py
View File

@ -103,7 +103,12 @@ HiddenServicePort 80 127.0.0.1:''' + str(self.hsPort) + '''
int(pidN)
except:
return
try:
os.kill(int(pidN), signal.SIGTERM)
os.remove('data/torPid.txt')
except ProcessLookupError:
pass
except FileNotFoundError:
pass
return

18
onionr/onionr.py
View File

@ -47,7 +47,6 @@ class Onionr:
if os.path.exists('dev-enabled'):
self._developmentMode = True
logger.set_level(logger.LEVEL_DEBUG)
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
else:
self._developmentMode = False
logger.set_level(logger.LEVEL_INFO)
@ -128,6 +127,7 @@ class Onionr:
'addmessage': self.addMessage,
'add-msg': self.addMessage,
'add-message': self.addMessage,
'pm': self.sendEncrypt,
'gui': self.openGUI,
'addpeer': self.addPeer,
'add-peer': self.addPeer
@ -150,6 +150,19 @@ class Onionr:
logger.info('Onionr ' + ONIONR_VERSION + ' (' + platform.machine() + ') : API v' + API_VERSION)
logger.info('Running on ' + platform.platform() + ' ' + platform.release())
def sendEncrypt(self):
'''Create a private message and send it'''
while True:
peer = logger.readline('Peer to send to: ')
if self.onionrUtils.validateID(peer):
break
else:
logger.error('Invalid peer ID')
message = logger.readline("Enter a message: ")
logger.info("Sending message to " + peer)
self.onionrUtils.sendPM(peer, message)
def openGUI(self):
gui.OnionrGUI(self.onionrCore)
@ -197,11 +210,14 @@ class Onionr:
def daemon(self):
''' Start the Onionr communication daemon '''
if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
if self._developmentMode:
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
net = NetController(self.config['CLIENT']['PORT'])
logger.info('Tor is starting...')
if not net.startTor():
sys.exit(1)
logger.info('Started Tor .onion service: ' + logger.colors.underline + net.myID)
logger.info('Our Public key: ' + self.onionrCore._crypto.pubKey)
time.sleep(1)
subprocess.Popen(["./communicator.py", "run", str(net.socksPort)])
logger.debug('Started communicator')

44
onionr/onionrcrypto.py
View File

@ -17,17 +17,51 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import nacl
import nacl.signing, nacl.encoding, nacl.public, os
class OnionrCrypto:
def __init__(self):
def __init__(self, coreInstance):
self._core = coreInstance
self._keyFile = 'data/keys.txt'
self.pubKey = None
self.privKey = None
# Load our own pub/priv Ed25519 keys, gen & save them if they don't exist
if os.path.exists(self._keyFile):
with open('data/keys.txt', 'r') as keys:
keys = keys.read().split(',')
self.pubKey = keys[0]
self.privKey = keys[1]
else:
keys = self.generatePubKey()
self.pubKey = keys[0]
self.privKey = keys[1]
with open(self._keyFile, 'w') as keyfile:
keyfile.write(self.pubKey + ',' + self.privKey)
return
def symmetricPeerEncrypt(self, data, key):
def pubKeyEncrypt(self, data, peer):
'''Encrypt to a peers public key (Curve25519, taken from Ed25519 pubkey)'''
return
def symmetricPeerDecrypt(self, data, key):
def pubKeyEncrypt(self, data, peer):
'''pubkey decrypt (Curve25519, taken from Ed25519 pubkey)'''
return
def rsaEncrypt(self, peer, data):
def symmetricPeerEncrypt(self, data):
'''Salsa20 encrypt data to peer (with mac)'''
return
def symmetricPeerDecrypt(self, data, peer):
'''Salsa20 decrypt data from peer (with mac)'''
return
def generateSymmetric(self, data, peer):
'''Generate symmetric key'''
return
def generatePubKey(self):
'''Generate a Ed25519 public key pair, return tuple of base64encoded pubkey, privkey'''
private_key = nacl.signing.SigningKey.generate()
public_key = private_key.verify_key.encode(encoder=nacl.encoding.Base32Encoder())
return (public_key.decode(), private_key.encode(encoder=nacl.encoding.Base32Encoder()).decode())

30
onionr/onionrutils.py
View File

@ -18,7 +18,8 @@
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
# Misc functions that do not fit in the main api, but are useful
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger, sqlite3
import getpass, sys, requests, configparser, os, socket, hashlib, logger, sqlite3
import nacl.signing, nacl.encoding
if sys.version_info < (3, 6):
try:
import sha3
@ -93,19 +94,6 @@ class OnionrUtils:
else:
return True
def exportMyPubkey(self):
'''
Export our PGP key if it exists
'''
if not os.path.exists(self.fingerprintFile):
raise Exception("No fingerprint found, cannot export our PGP key.")
gpg = gnupg.GPG(homedir='./data/pgp/')
with open(self.fingerprintFile,'r') as f:
fingerprint = f.read()
ascii_armored_public_keys = gpg.export_keys(fingerprint)
return ascii_armored_public_keys
def getBlockDBHash(self):
'''
Return a sha3_256 hash of the blocks DB
@ -154,9 +142,21 @@ class OnionrUtils:
return retVal
def validatePubKey(self, key):
'''Validate if a string is a valid base32 encoded Ed25519 key'''
retVal = False
try:
nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
except nacl.exceptions.ValueError:
pass
else:
retVal = True
return retVal
def validateID(self, id):
'''
Validate if a user ID is a valid tor or i2p hidden service
Validate if an address is a valid tor or i2p hidden service
'''
idLength = len(id)
retVal = True

29
onionr/tests.py
View File

@ -54,7 +54,7 @@ class OnionrTests(unittest.TestCase):
myCore = core.Core()
if not os.path.exists('data/peers.db'):
myCore.createPeerDB()
if myCore.addPeer('2ks5c5bm6zk3ejqg.onion') and not myCore.addPeer('invalidpeer.onion'):
if myCore.addPeer('6M5MXL237OK57ITHVYN5WGHANPGOMKS5C3PJLHBBNKFFJQOIDOJA====') and not myCore.addPeer('NFXHMYLMNFSAU==='):
self.assertTrue(True)
else:
self.assertTrue(False)
@ -85,33 +85,6 @@ class OnionrTests(unittest.TestCase):
else:
self.assertTrue(False)
def testPGPGen(self):
logger.debug('--------------------------')
logger.info('Running PGP key generation test...')
if os.path.exists('data/pgp/'):
self.assertTrue(True)
else:
import core, netcontroller
myCore = core.Core()
net = netcontroller.NetController(1337)
net.startTor()
torID = open('data/hs/hostname').read()
myCore.generateMainPGP(torID)
if os.path.exists('data/pgp/'):
self.assertTrue(True)
def testHMACGen(self):
logger.debug('--------------------------')
logger.info('Running HMAC generation test...')
# Test if hmac key generation is working
import core
myCore = core.Core()
key = myCore.generateHMAC()
if len(key) > 10:
self.assertTrue(True)
else:
self.assertTrue(False)
def testQueue(self):
logger.debug('--------------------------')
logger.info('Running daemon queue test...')

12
readme.md
View File

@ -2,17 +2,27 @@
[![Build Status](https://travis-ci.org/beardog108/onionr.svg?branch=master)](https://travis-ci.org/beardog108/onionr)
P2P microblogging platform and social network, using Tor & I2P.
P2P platform, using Tor & I2P.
Major work in progress.
***THIS SOFTWARE IS NOT USABLE OR SAFE YET.***
**Roadmap/features:**
* [X] Fully p2p/decentralized, no trackers or other single points of failure
* [X] High level of anonymity
* [ ] End to end encryption where applicable
* [X] Optional non-encrypted blocks, useful for blog posts or public file sharing
* [ ] Easy API system for integration to websites
# Development
This software is in heavy development. If for some reason you want to get involved, get in touch first.
**Onionr API and functionality is subject to non-backwards compatible change during development**
## Disclaimer
The Tor Project, I2P developers, and anyone else do not own, create, or endorse this project, and are not otherwise involved.

7
requirements.txt
View File

@ -1,10 +1,7 @@
PyNaCl==1.2.1
gnupg==2.3.1
requests==2.12.4
Flask==0.12.2
requests==2.18.4
urllib3==1.22
simple_crypt==4.1.7
urllib3==1.19.1
sha3==0.2.1
pycrypto==2.6.1
pynacl==1.2.1
PySocks==1.6.8

5
reset.sh Executable file
View File

@ -0,0 +1,5 @@
#!/bin/bash
echo "RESETING ONIONR"
rm onionr/data/blocks/*.dat
rm onionr/data/peers.db
rm onionr/data/blocks.db