kev
/
Onionr
1
0
Fork 0
Code Issues 24 Pull Requests Projects Releases 2 Wiki Activity

Merge branch 'crypto'

This commit is contained in:
Kevin Froman 2018-04-01 01:41:24 -05:00
parent 980ef3c200 49b7efacff
commit e3ebe5c2e4
29 changed files with 1946 additions and 433 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitignore vendored
View File

@ -6,3 +6,6 @@ onionr/*.pyc
onionr/*.log
onionr/data/hs/hostname
onionr/data/*
onionr/data-backup/*
onionr/gnupg/*
run.sh

3
.gitmodules vendored Normal file
View File

@ -0,0 +1,3 @@
[submodule "onionr/bitpeer"]
path = onionr/bitpeer
url = https://github.com/beardog108/bitpeer.py

2
.travis.yml
View File

@ -5,4 +5,4 @@ python:
install:
- sudo apt install gnupg tor
- pip install -r requirements.txt
script: ./test.sh
script: make test

29
Makefile Normal file
View File

@ -0,0 +1,29 @@
.DEFAULT_GOAL := setup
setup:
sudo pip3 install -r requirements.txt
install:
sudo rm -rf /usr/share/onionr/
sudo rm -f /usr/bin/onionr
sudo cp -rp ./onionr /usr/share/onionr
sudo sh -c "echo \"#!/bin/sh\ncd /usr/share/onionr/\n./onionr.py \\\"\\\$$@\\\"\" > /usr/bin/onionr"
sudo chmod +x /usr/bin/onionr
sudo chown -R `whoami` /usr/share/onionr/
uninstall:
sudo rm -rf /usr/share/onionr
sudo rm -f /usr/bin/onionr
test:
@rm -rf onionr/data-backup
@mv onionr/data onionr/data-backup | true > /dev/null 2>&1
-@cd onionr; ./tests.py
@rm -rf onionr/data
@mv onionr/data-backup onionr/data | true > /dev/null 2>&1
reset:
rm -f onionr/data/blocks/*.dat | true > /dev/null 2>&1
rm -f onionr/data/peers.db | true > /dev/null 2>&1
rm -f onionr/data/blocks.db | true > /dev/null 2>&1
rm -f onionr/data/address.db | true > /dev/null 2>&1

6
docs/api.md
View File

@ -1,3 +1,9 @@
BLOCK HEADERS (simple ID system to identify block type)
-----------------------------------------------
-crypt- (encrypted block)
-bin- (binary file)
-txt- (plaintext)
HTTP API
------------------------------------------------
/client/ (Private info, not publicly accessible)

77
docs/onionr-draft.md
View File

@ -1,72 +1,51 @@
# Onionr Protocol Spec
# Onionr Protocol Spec v2
A social network/microblogging platform for Tor & I2P
Draft Dec 25 2017
A P2P platform for Tor & I2P
# Overview
Onionr is an encrypted microblogging & mailing system designed in the spirit of Twitter.
There are no central servers and all traffic is peer to peer by default (routed via Tor or I2P).
User IDs are simply Tor onion service/I2P host id + PGP fingerprint.
Clients consolidate feeds from peers into 1 “timeline” using RSS format.
Private messages are only accessible by the intended peer based on the PGP id.
Onionr is not intended to be a replacement for Ricochet, OnionShare, or Briar.
All traffic is over onion/I2P because if only some was, then that would make that traffic inherently suspicious.
User IDs are simply Tor onion service/I2P host id + Ed25519 key fingerprint.
Private blocks are only able to be read by the intended peer.
All traffic is over Tor/I2P, connecting only to Tor onion and I2P hidden services.
## Goals:
• Selective sharing of information with friends & public
• Selective sharing of information
• Secure & semi-anonymous direct messaging
• Forward secrecy
• Defense in depth
• Data should be secure for years to come, quantum safe (though not necessarily every “layer”)
• Data should be secure for years to come
• Decentralization
* Avoid browser-based exploits that plague similar software
* Avoid timing attacks & unexpected metadata leaks
## Assumptions:
• Tor & I2P’s transport protocols & AES-256 are not broken, sha3-512 2nd preimage attacks will remain infeasible indefinitely
• All traffic is logged indefinitely by powerful adversaries
## Protocol
Clients MUST use HTTP(s) to communicate with one another to maintain compatibility cross platform. HTTPS is recommended, but HTTP is acceptable because Tor & I2P provide transport layer security.
Onionr nodes use HTTP (over Tor/I2P) to exchange keys, metadata, and blocks. Blocks are identified by their sha3_256 hash. Nodes sync a table of blocks hashes and attempt to download blocks they do not yet have from random peers.
Blocks may be encrypted using Curve25519.
## Connections
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange PGP public keys and then generate a shared AES-SHA3-512 HMAC token. These keys are stored in a peer database until expiry.
HMAC tokens are regenerated either every X many communications with a peer or every X minutes. Every 10MB or every 2 hours is a recommended default.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends:
* Encrypted ‘friends only’ posts to one another
* Usually less strict rate & storage limits
* OPTIONALLY sign one another’s keys. Users may not want to do this in order to avoid exposing their entire friends list.
• Strangers:
* Used for storage of encrypted or public information
* Can only read public posts
* Usually stricter rate & storage limits
## Data Storage/Delivery
Posts (public or friends only) are stored across the network.
Private messages SHOULD be delivered directly if both peers are online, otherwise stored in the network.
Data SHOULD be stored in an entirely encrypted state when a client is offline, including metadata. Data SHOULD be stored in a minimal size with garbage data to ensure some level of plausible deniablity.
Data SHOULD be stored as long as the node’s user prefers and only erased once disk quota is reached due to new data.
Posts
Posts can contain text and images. All posts MUST be time stamped.
Images SHOULD not be displayed by non-friends by default, to prevent unwanted viewing of offensive material & to reduce attack surface.
All received posts must be verified to be stored and/or displayed to the user.
When a node first comes online, it attempts to bootstrap using a default list provided by a client.
When two peers connect, they exchange Ed25519 keys (if applicable) then Salsa20 keys.
All data being transfered MUST be encrypted to the end node receiving the data, then the data MUST be encrypted the node(s) transporting/storing the data,
Salsa20 keys are regenerated either every X many communications with a peer or every X minutes.
Posts have two settings:
• Friends only:
◦ Posts MUST be encrypted to all trusted peers via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary RSA key for use in every post (or message) is exchanged every X many configured post (or message), for use in addition with PGP and the HMAC.
• Public:
◦ Posts MUST be PGP signed, and MUST NOT use any encryption.
## Private Messages
Every 100kb or every 2 hours is a recommended default.
Private messages are messages that can have attached images. They MUST be encrypted via AES256-HMAC-SHA256 and PGP signed (signed before encryption) and time stamped to prevent replaying. A temporary EdDSA key for use in every message is exchanged every X many configured messages (or posts), for use in addition with PGP and the HMAC.
When both peers are online messages SHOULD be dispatched directly between peers.
All messages must be verified prior to being displayed.
All valid requests with HMAC should be recorded until used HMAC's expiry to prevent replay attacks.
Peer Types
* Friends:
* Encrypted ‘friends only’ posts to one another
* Usually less strict rate & storage limits
* Strangers:
* Used for storage of encrypted or public information
* Can only read public posts
* Usually stricter rate & storage limits
Clients SHOULD allow configurable message padding.
## Spam mitigation
To send or receive data, a node can optionally request that the other node generate a hash that when in hexadecimal representation contains a random string at a random location in the string. Clients will configure what difficulty to request, and what difficulty is acceptable for themselves to perform. Difficulty should correlate with recent network & disk usage and data size. Friends can be configured to have less strict (to non existent) limits, separately from strangers. (proof of work).
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.
Rate limits can be strict, as Onionr is not intended to be an instant messaging application.

BIN
docs/onionr-logo.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 5.2 KiB

81
onionr/api.py
View File

@ -20,45 +20,51 @@
import flask
from flask import request, Response, abort
from multiprocessing import Process
import configparser, sys, random, threading, hmac, hashlib, base64, time, math, gnupg, os, logger
import sys, random, threading, hmac, hashlib, base64, time, math, os, logger, config
from core import Core
import onionrutils
import onionrutils, onionrcrypto
class API:
''' Main http api (flask)'''
'''
Main HTTP API (Flask)
'''
def validateToken(self, token):
'''
Validate if the client token (hmac) matches the given token
Validate that the client token (hmac) matches the given token
'''
if self.clientToken != token:
return False
else:
return True
def __init__(self, config, debug):
''' Initialize the api server, preping variables for later use
This initilization defines all of the API entry points and handlers for the endpoints and errors
This also saves the used host (random localhost IP address) to the data folder in host.txt
def __init__(self, debug):
'''
if os.path.exists('dev-enabled'):
Initialize the api server, preping variables for later use
This initilization defines all of the API entry points and handlers for the endpoints and errors
This also saves the used host (random localhost IP address) to the data folder in host.txt
'''
config.reload()
if config.get('devmode', True):
self._developmentMode = True
logger.set_level(logger.LEVEL_DEBUG)
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
else:
self._developmentMode = False
logger.set_level(logger.LEVEL_INFO)
self.config = config
self.debug = debug
self._privateDelayTime = 3
self._core = Core()
self._crypto = onionrcrypto.OnionrCrypto(self._core)
self._utils = onionrutils.OnionrUtils(self._core)
app = flask.Flask(__name__)
bindPort = int(self.config['CLIENT']['PORT'])
bindPort = int(config.get('client')['port'])
self.bindPort = bindPort
self.clientToken = self.config['CLIENT']['CLIENT HMAC']
logger.debug('Your HMAC token: ' + logger.colors.underline + self.clientToken)
self.clientToken = config.get('client')['client_hmac']
if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
logger.debug('Your HMAC token: ' + logger.colors.underline + self.clientToken)
if not debug and not self._developmentMode:
hostNums = [random.randint(1, 255), random.randint(1, 255), random.randint(1, 255)]
@ -72,9 +78,10 @@ class API:
@app.before_request
def beforeReq():
'''
Simply define the request as not having yet failed, before every request.
Simply define the request as not having yet failed, before every request.
'''
self.requestFailed = False
return
@app.after_request
@ -87,6 +94,7 @@ class API:
resp.headers["Content-Security-Policy"] = "default-src 'none'"
resp.headers['X-Frame-Options'] = 'deny'
resp.headers['X-Content-Type-Options'] = "nosniff"
return resp
@app.route('/client/')
@ -112,6 +120,7 @@ class API:
elapsed = endTime - startTime
if elapsed < self._privateDelayTime:
time.sleep(self._privateDelayTime - elapsed)
return resp
@app.route('/public/')
@ -125,14 +134,14 @@ class API:
pass
elif action == 'ping':
resp = Response("pong!")
elif action == 'setHMAC':
pass
elif action == 'getHMAC':
resp = Response(self._crypto.generateSymmetric())
elif action == 'getSymmetric':
resp = Response(self._crypto.generateSymmetric())
elif action == 'getDBHash':
resp = Response(self._utils.getBlockDBHash())
elif action == 'getBlockHashes':
resp = Response(self._core.getBlockList())
elif action == 'getPGP':
resp = Response(self._utils.exportMyPubkey())
# setData should be something the communicator initiates, not this api
elif action == 'getData':
resp = self._core.getData(data)
@ -140,6 +149,16 @@ class API:
abort(404)
resp = ""
resp = Response(resp)
elif action == 'pex':
response = ','.join(self._core.listAdders())
if len(response) == 0:
response = 'none'
resp = Response(response)
elif action == 'kex':
response = ','.join(self._core.listPeers())
if len(response) == 0:
response = 'none'
resp = Response(response)
else:
resp = Response("")
@ -149,26 +168,36 @@ class API:
def notfound(err):
self.requestFailed = True
resp = Response("")
#resp.headers = getHeaders(resp)
return resp
@app.errorhandler(403)
def authFail(err):
self.requestFailed = True
resp = Response("403")
return resp
@app.errorhandler(401)
def clientError(err):
self.requestFailed = True
resp = Response("Invalid request")
return resp
if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
logger.info('Starting client on ' + self.host + ':' + str(bindPort) + '...')
logger.debug('Client token: ' + logger.colors.underline + self.clientToken)
app.run(host=self.host, port=bindPort, debug=True, threaded=True)
try:
app.run(host=self.host, port=bindPort, debug=True, threaded=True)
except Exception as e:
logger.error(str(e))
logger.fatal('Failed to start client on ' + self.host + ':' + str(bindPort) + ', exiting...')
exit(1)
def validateHost(self, hostType):
''' Validate various features of the request including:
'''
Validate various features of the request including:
If private (/client/), is the host header local?
If public (/public/), is the host header onion or i2p?

1
onionr/bitpeer Submodule

@ -0,0 +1 @@
Subproject commit a74e826e9c69e643ead7950f9f76a05ab8664ddc

11
onionr/btc.py
View File

@ -20,8 +20,9 @@
from bitpeer.node import *
from bitpeer.storage.shelve import ShelveStorage
import logging, time
import socks, sys
class OnionrBTC:
def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613):
def __init__(self, lastBlock='00000000000000000021ee6242d08e3797764c9258e54e686bc2afff51baf599', lastHeight=510613, torP=9050):
stream = logging.StreamHandler()
logger = logging.getLogger('halfnode')
logger.addHandler(stream)
@ -29,9 +30,15 @@ class OnionrBTC:
LASTBLOCK = lastBlock
LASTBLOCKINDEX = lastHeight
self.node = Node ('BTC', ShelveStorage ('./btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX)
self.node = Node ('BTC', ShelveStorage ('data/btc-blocks.db'), lastblockhash=LASTBLOCK, lastblockheight=LASTBLOCKINDEX, torPort=torP)
self.node.bootstrap ()
self.node.connect ()
self.node.loop ()
if __name__ == "__main__":
torPort = int(sys.argv[1])
bitcoin = OnionrBTC(torPort)
while True:
print(bitcoin.node.getBlockHash(bitcoin.node.getLastBlockHeight())) # Using print on purpose, do not change to logger
time.sleep(5)

23
onionr/colors.py
View File

@ -1,23 +0,0 @@
'''
Simply define terminal control codes (mainly colors)
'''
class Colors:
def __init__(self):
'''
PURPLE='\033[95m'
BLUE='\033[94m'
GREEN='\033[92m'
YELLOW='\033[93m'
RED='\033[91m'
BOLD='\033[1m'
UNDERLINE='\033[4m'
RESET="\x1B[m"
'''
self.PURPLE='\033[95m'
self.BLUE='\033[94m'
self.GREEN='\033[92m'
self.YELLOW='\033[93m'
self.RED='\033[91m'
self.BOLD='\033[1m'
self.UNDERLINE='\033[4m'
self.RESET="\x1B[m"

221
onionr/communicator.py
View File

@ -19,93 +19,144 @@ and code to operate as a daemon, getting commands from the command queue databas
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sqlite3, requests, hmac, hashlib, time, sys, os, logger
import core, onionrutils
import sqlite3, requests, hmac, hashlib, time, sys, os, math, logger, urllib.parse, random
import core, onionrutils, onionrcrypto, onionrproofs, btc, config, onionrplugins as plugins
class OnionrCommunicate:
def __init__(self, debug, developmentMode):
''' OnionrCommunicate
This class handles communication with nodes in the Onionr network.
'''
OnionrCommunicate
This class handles communication with nodes in the Onionr network.
'''
self._core = core.Core()
self._utils = onionrutils.OnionrUtils(self._core)
self._crypto = onionrcrypto.OnionrCrypto(self._core)
'''
logger.info('Starting Bitcoin Node... with Tor socks port:' + str(sys.argv[2]))
try:
self.bitcoin = btc.OnionrBTC(torP=int(sys.argv[2]))
except _gdbm.error:
pass
logger.info('Bitcoin Node started, on block: ' + self.bitcoin.node.getBlockHash(self.bitcoin.node.getLastBlockHeight()))
'''
#except:
#logger.fatal('Failed to start Bitcoin Node, exiting...')
#exit(1)
blockProcessTimer = 0
blockProcessAmount = 5
heartBeatTimer = 0
heartBeatRate = 10
heartBeatRate = 5
pexTimer = 5 # How often we should check for new peers
pexCount = 0
logger.debug('Communicator debugging enabled.')
torID = open('data/hs/hostname').read()
# get our own PGP fingerprint
fingerprintFile = 'data/own-fingerprint.txt'
if not os.path.exists(fingerprintFile):
self._core.generateMainPGP(torID)
with open(fingerprintFile,'r') as f:
self.pgpOwnFingerprint = f.read()
logger.info('My PGP fingerprint is ' + logger.colors.underline + self.pgpOwnFingerprint + logger.colors.reset + logger.colors.fg.green + '.')
self.peerData = {} # Session data for peers (recent reachability, speed, etc)
if os.path.exists(self._core.queueDB):
self._core.clearDaemonQueue()
# Loads in and starts the enabled plugins
plugins.reload()
while True:
command = self._core.daemonQueue()
# Process blocks based on a timer
blockProcessTimer += 1
heartBeatTimer += 1
pexCount += 1
if pexTimer == pexCount:
self.getNewPeers()
pexCount = 0
if heartBeatRate == heartBeatTimer:
logger.debug('Communicator heartbeat')
heartBeatTimer = 0
if blockProcessTimer == blockProcessAmount:
self.lookupBlocks()
self._core.processBlocks()
self.processBlocks()
blockProcessTimer = 0
#logger.debug('Communicator daemon heartbeat')
if command != False:
if command[0] == 'shutdown':
logger.warn('Daemon recieved exit command.')
logger.info('Daemon recieved exit command.')
break
time.sleep(1)
return
def getRemotePeerKey(self, peerID):
'''This function contacts a peer and gets their main PGP key.
This is safe because Tor or I2P is used, but it does not ensure that the person is who they say they are
'''
url = 'http://' + peerID + '/public/?action=getPGP'
r = requests.get(url, headers=headers)
response = r.text
return response
def shareHMAC(self, peerID, key):
'''This function shares an HMAC key to a peer
'''
return
def getPeerProof(self, peerID):
'''This function gets the current peer proof requirement'''
return
def sendPeerProof(self, peerID, data):
'''This function sends the proof result to a peer previously fetched with getPeerProof'''
def getNewPeers(self):
'''
Get new peers
'''
peersCheck = 5 # Amount of peers to ask for new peers + keys
peersChecked = 0
peerList = list(self._core.listAdders()) # random ordered list of peers
newKeys = []
newAdders = []
if len(peerList) > 0:
maxN = len(peerList) - 1
else:
peersCheck = 0
maxN = 0
if len(peerList) > peersCheck:
peersCheck = len(peerList)
while peersCheck > peersChecked:
i = random.randint(0, maxN)
logger.info('Using ' + peerList[i] + ' to find new peers')
try:
newAdders = self.performGet('pex', peerList[i], skipHighFailureAddress=True)
self._utils.mergeAdders(newAdders)
except requests.exceptions.ConnectionError:
logger.info(peerList[i] + ' connection failed')
continue
else:
try:
logger.info('Using ' + peerList[i] + ' to find new keys')
newKeys = self.performGet('kex', peerList[i], skipHighFailureAddress=True)
# TODO: Require keys to come with POW token (very large amount of POW)
self._utils.mergeKeys(newKeys)
except requests.exceptions.ConnectionError:
logger.info(peerList[i] + ' connection failed')
continue
else:
peersChecked += 1
return
def lookupBlocks(self):
'''Lookup blocks and merge new ones'''
peerList = self._core.listPeers()
'''
Lookup blocks and merge new ones
'''
peerList = self._core.listAdders()
blocks = ''
for i in peerList:
lastDB = self._core.getPeerInfo(i, 'blockDBHash')
lastDB = self._core.getAddressInfo(i, 'DBHash')
if lastDB == None:
logger.debug('Fetching hash from ' + i + ' No previous known.')
else:
logger.debug('Fetching hash from ' + i + ', ' + lastDB + ' last known')
logger.debug('Fetching hash from ' + str(i) + ', ' + lastDB + ' last known')
currentDB = self.performGet('getDBHash', i)
if currentDB != False:
logger.debug(i + " hash db (from request): " + currentDB)
else:
logger.warn("Error getting hash db status for " + i)
if currentDB != False:
if lastDB != currentDB:
logger.debug('Fetching hash from ' + i + ' - ' + currentDB + ' current hash.')
blocks += self.performGet('getBlockHashes', i)
if currentDB != lastDB:
if self._utils.validateHash(currentDB):
self._core.setPeerInfo(i, "blockDBHash", currentDB)
else:
logger.warn("Peer " + i + " returned malformed hash")
self._core.setAddressInfo(i, "DBHash", currentDB)
if len(blocks.strip()) != 0:
logger.debug('BLOCKS:' + blocks)
blockList = blocks.split('\n')
for i in blockList:
if len(i.strip()) == 0:
continue
if self._utils.hasBlock(i):
continue
logger.debug('Exchanged block (blockList): ' + i)
if not self._utils.validateHash(i):
# skip hash if it isn't valid
@ -114,31 +165,100 @@ class OnionrCommunicate:
else:
logger.debug('Adding ' + i + ' to hash database...')
self._core.addToBlockDB(i)
return
def performGet(self, action, peer, data=None, type='tor'):
'''Performs a request to a peer through Tor or i2p (currently only tor)'''
def processBlocks(self):
'''
Work with the block database and download any missing blocks
This is meant to be called from the communicator daemon on its timer.
'''
for i in self._core.getBlockList(True).split("\n"):
if i != "":
logger.warn('UNSAVED BLOCK: ' + i)
data = self.downloadBlock(i)
return
def downloadBlock(self, hash):
'''
Download a block from random order of peers
'''
peerList = self._core.listAdders()
blocks = ''
for i in peerList:
hasher = hashlib.sha3_256()
data = self.performGet('getData', i, hash)
if data == False or len(data) > 10000000:
continue
hasher.update(data.encode())
digest = hasher.hexdigest()
if type(digest) is bytes:
digest = digest.decode()
if digest == hash.strip():
self._core.setData(data)
if data.startswith('-txt-'):
self._core.setBlockType(hash, 'txt')
logger.info('Successfully obtained data for ' + hash)
if len(data) < 120:
logger.debug('Block text:\n' + data)
else:
logger.warn("Failed to validate " + hash)
return
def urlencode(self, data):
'''
URL encodes the data
'''
return urllib.parse.quote_plus(data)
def performGet(self, action, peer, data=None, skipHighFailureAddress=False, peerType='tor'):
'''
Performs a request to a peer through Tor or i2p (currently only Tor)
'''
if not peer.endswith('.onion') and not peer.endswith('.onion/'):
raise PeerError('Currently only Tor .onion peers are supported. You must manually specify .onion')
# Store peer in peerData dictionary (non permanent)
if not peer in self.peerData:
self.peerData[peer] = {'connectCount': 0, 'failCount': 0, 'lastConnectTime': math.floor(time.time())}
socksPort = sys.argv[2]
'''We use socks5h to use tor as DNS'''
proxies = {'http': 'socks5h://127.0.0.1:' + str(socksPort), 'https': 'socks5h://127.0.0.1:' + str(socksPort)}
proxies = {'http': 'socks5://127.0.0.1:' + str(socksPort), 'https': 'socks5://127.0.0.1:' + str(socksPort)}
headers = {'user-agent': 'PyOnionr'}
url = 'http://' + peer + '/public/?action=' + action
url = 'http://' + peer + '/public/?action=' + self.urlencode(action)
if data != None:
url = url + '&data=' + data
url = url + '&data=' + self.urlencode(data)
try:
r = requests.get(url, headers=headers, proxies=proxies)
if skipHighFailureAddress and self.peerData[peer]['failCount'] > 10:
retData = False
logger.debug('Skipping ' + peer + ' because of high failure rate')
else:
logger.debug('Contacting ' + peer + ' on port ' + socksPort)
r = requests.get(url, headers=headers, proxies=proxies, timeout=(15, 30))
retData = r.text
except requests.exceptions.RequestException as e:
logger.warn(action + " failed with peer " + peer + ": " + str(e))
return False
return r.text
retData = False
if not retData:
self.peerData[peer]['failCount'] += 1
else:
self.peerData[peer]['connectCount'] += 1
self.peerData[peer]['lastConnectTime'] = math.floor(time.time())
return retData
shouldRun = False
debug = True
developmentMode = False
if os.path.exists('dev-enabled'):
if config.get('devmode', True):
developmentMode = True
try:
if sys.argv[1] == 'run':
@ -149,4 +269,5 @@ if shouldRun:
try:
OnionrCommunicate(debug, developmentMode)
except KeyboardInterrupt:
sys.exit(1)
pass

114
onionr/config.py Normal file
View File

@ -0,0 +1,114 @@
'''
Onionr - P2P Microblogging Platform & Social network
This file deals with configuration management.
'''
'''
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import os, json, logger
_configfile = os.path.abspath('data/config.json')
_config = {}
def get(key, default = None):
'''
Gets the key from configuration, or returns `default`
'''
if is_set(key):
return get_config()[key]
return default
def set(key, value = None, savefile = False):
'''
Sets the key in configuration to `value`
'''
global _config
if value is None:
del _config[key]
else:
_config[key] = value
if savefile:
save()
def is_set(key):
return key in get_config() and not get_config()[key] is None
def check():
'''
Checks if the configuration file exists, creates it if not
'''
try:
if not os.path.exists(os.path.dirname(get_config_file())):
os.path.mkdirs(os.path.dirname(get_config_file()))
if not os.path.isfile(get_config_file()):
open(get_config_file(), 'a', encoding="utf8").close()
save()
except:
logger.warn('Failed to check configuration file.')
def save():
'''
Saves the configuration data to the configuration file
'''
check()
try:
with open(get_config_file(), 'w', encoding="utf8") as configfile:
json.dump(get_config(), configfile, indent=2, sort_keys=True)
except:
logger.warn('Failed to write to configuration file.')
def reload():
'''
Reloads the configuration data in memory from the file
'''
check()
try:
with open(get_config_file(), 'r', encoding="utf8") as configfile:
set_config(json.loads(configfile.read()))
except:
logger.warn('Failed to parse configuration file.')
def get_config():
'''
Gets the entire configuration as an array
'''
return _config
def set_config(config):
'''
Sets the configuration to the array in arguments
'''
global _config
_config = config
def get_config_file():
'''
Returns the absolute path to the configuration file
'''
return _configfile
def set_config_file(configfile):
'''
Sets the path to the configuration file
'''
global _configfile
_configfile = os.abs.abspath(configfile)

358
onionr/core.py
View File

@ -1,7 +1,7 @@
'''
Onionr - P2P Microblogging Platform & Social network
Core Onionr library, useful for external programs. Handles peer processing and cryptography.
Core Onionr library, useful for external programs. Handles peer & data processing
'''
'''
This program is free software: you can redistribute it and/or modify
@ -17,12 +17,12 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sqlite3, os, sys, time, math, gnupg, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
from Crypto.Cipher import AES
from Crypto import Random
import sqlite3, os, sys, time, math, base64, tarfile, getpass, simplecrypt, hashlib, nacl, logger
#from Crypto.Cipher import AES
#from Crypto import Random
import netcontroller
import onionrutils
import onionrutils, onionrcrypto, btc
if sys.version_info < (3, 6):
try:
@ -34,106 +34,153 @@ if sys.version_info < (3, 6):
class Core:
def __init__(self):
'''
Initialize Core Onionr library
Initialize Core Onionr library
'''
self.queueDB = 'data/queue.db'
self.peerDB = 'data/peers.db'
self.ownPGPID = ''
self.blockDB = 'data/blocks.db'
self.blockDataLocation = 'data/blocks/'
self._utils = onionrutils.OnionrUtils(self)
self.addressDB = 'data/address.db'
if not os.path.exists('data/'):
os.mkdir('data/')
if not os.path.exists('data/blocks/'):
os.mkdir('data/blocks/')
if not os.path.exists(self.blockDB):
self.createBlockDB()
self._utils = onionrutils.OnionrUtils(self)
# Initialize the crypto object
self._crypto = onionrcrypto.OnionrCrypto(self)
return
def generateMainPGP(self, myID):
''' Generate the main PGP key for our client. Should not be done often.
Uses own PGP home folder in the data/ directory. '''
# Generate main pgp key
gpg = gnupg.GPG(homedir='./data/pgp/')
input_data = gpg.gen_key_input(key_type="RSA", key_length=1024, name_real=myID, name_email='anon@onionr', testing=True)
#input_data = gpg.gen_key_input(key_type="RSA", key_length=1024)
key = gpg.gen_key(input_data)
logger.info("Generating PGP key, this will take some time..")
while key.status != "key created":
time.sleep(0.5)
print(key.status)
logger.info("Finished generating PGP key")
# Write the key
myFingerpintFile = open('data/own-fingerprint.txt', 'w')
myFingerpintFile.write(key.fingerprint)
myFingerpintFile.close()
return
def addPeer(self, peerID, name=''):
''' Add a peer by their ID, with an optional name, to the peer database.'''
''' DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion. '''
'''
Adds a public key to the key database (misleading function name)
DOES NO SAFETY CHECKS if the ID is valid, but prepares the insertion
'''
# This function simply adds a peer to the DB
if not self._utils.validateID(peerID):
if not self._utils.validatePubKey(peerID):
return False
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
t = (peerID, name, 'unknown')
c.execute('insert into peers (id, name, dateSeen) values(?, ?, ?);', t)
c.execute('INSERT INTO peers (id, name, dateSeen) VALUES(?, ?, ?);', t)
conn.commit()
conn.close()
return True
def addAddress(self, address):
'''Add an address to the address database (only tor currently)'''
if self._utils.validateID(address):
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
t = (address, 1)
c.execute('INSERT INTO adders (address, type) VALUES(?, ?);', t)
conn.commit()
conn.close()
return True
else:
return False
def removeAddress(self, address):
'''Remove an address from the address database'''
if self._utils.validateID(address):
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
t = (address,)
c.execute('Delete from adders where address=?;', t)
conn.commit()
conn.close()
return True
else:
return False
def createAddressDB(self):
'''
Generate the address database
types:
1: I2P b32 address
2: Tor v2 (like facebookcorewwwi.onion)
3: Tor v3
'''
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
c.execute('''CREATE TABLE adders(
address text,
type int,
knownPeer text,
speed int,
success int,
DBHash text,
failure int
);
''')
conn.commit()
conn.close()
def createPeerDB(self):
'''
Generate the peer sqlite3 database and populate it with the peers table.
Generate the peer sqlite3 database and populate it with the peers table.
'''
# generate the peer database
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
c.execute('''
create table peers(
ID text not null,
name text,
pgpKey text,
hmacKey text,
blockDBHash text,
forwardKey text,
dateSeen not null,
bytesStored int,
trust int);
c.execute('''CREATE TABLE peers(
ID text not null,
name text,
adders text,
blockDBHash text,
forwardKey text,
dateSeen not null,
bytesStored int,
trust int);
''')
conn.commit()
conn.close()
return
def createBlockDB(self):
'''
Create a database for blocks
Create a database for blocks
hash - the hash of a block
dateReceived - the date the block was recieved, not necessarily when it was created
decrypted - if we can successfully decrypt the block (does not describe its current state)
dataObtained - if the data has been obtained for the block
hash - the hash of a block
dateReceived - the date the block was recieved, not necessarily when it was created
decrypted - if we can successfully decrypt the block (does not describe its current state)
dataType - data type of the block
dataFound - if the data has been found for the block
dataSaved - if the data has been saved for the block
'''
if os.path.exists(self.blockDB):
raise Exception("Block database already exists")
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
c.execute('''create table hashes(
c.execute('''CREATE TABLE hashes(
hash text not null,
dateReceived int,
decrypted int,
dataType text,
dataFound int,
dataSaved int
);
dataSaved int);
''')
conn.commit()
conn.close()
return
def addToBlockDB(self, newHash, selfInsert=False):
'''add a hash value to the block db (should be in hex format)'''
'''
Add a hash value to the block db
Should be in hex format!
'''
if not os.path.exists(self.blockDB):
raise Exception('Block db does not exist')
if self._utils.hasBlock(newHash):
return
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
currentTime = math.floor(time.time())
@ -141,41 +188,57 @@ class Core:
selfInsert = 1
else:
selfInsert = 0
data = (newHash, currentTime, 0, 0, selfInsert)
c.execute('INSERT into hashes values(?, ?, ?, ?, ?);', data)
data = (newHash, currentTime, 0, '', 0, selfInsert)
c.execute('INSERT INTO hashes VALUES(?, ?, ?, ?, ?, ?);', data)
conn.commit()
conn.close()
return
def getData(self,hash):
'''simply return the data associated to a hash'''
'''
Simply return the data associated to a hash
'''
try:
dataFile = open(self.blockDataLocation + hash + '.dat')
data = dataFile.read()
dataFile.close()
except FileNotFoundError:
data = False
return data
def setData(self, data):
'''set the data assciated with a hash'''
'''
Set the data assciated with a hash
'''
data = data.encode()
hasher = hashlib.sha3_256()
hasher.update(data)
dataHash = hasher.hexdigest()
if type(dataHash) is bytes:
dataHash = dataHash.decode()
blockFileName = self.blockDataLocation + dataHash + '.dat'
if os.path.exists(blockFileName):
raise Exception("Data is already set for " + dataHash)
pass # TODO: properly check if block is already saved elsewhere
#raise Exception("Data is already set for " + dataHash)
else:
blockFile = open(blockFileName, 'w')
blockFile.write(data.decode())
blockFile.close()
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
c.execute("UPDATE hashes SET dataSaved=1 WHERE hash = '" + dataHash + "';")
conn.commit()
conn.close()
return dataHash
def dataDirEncrypt(self, password):
'''
Encrypt the data directory on Onionr shutdown
Encrypt the data directory on Onionr shutdown
'''
# Encrypt data directory (don't delete it in this function)
if os.path.exists('data.tar'):
os.remove('data.tar')
tar = tarfile.open("data.tar", "w")
@ -186,12 +249,13 @@ class Core:
encrypted = simplecrypt.encrypt(password, tarData)
open('data-encrypted.dat', 'wb').write(encrypted)
os.remove('data.tar')
return
def dataDirDecrypt(self, password):
'''
Decrypt the data directory on startup
Decrypt the data directory on startup
'''
# Decrypt data directory
if not os.path.exists('data-encrypted.dat'):
return (False, 'encrypted archive does not exist')
data = open('data-encrypted.dat', 'rb').read()
@ -204,13 +268,15 @@ class Core:
tar = tarfile.open('data.tar')
tar.extractall()
tar.close()
return (True, '')
def daemonQueue(self):
'''
Gives commands to the communication proccess/daemon by reading an sqlite3 database
Gives commands to the communication proccess/daemon by reading an sqlite3 database
This function intended to be used by the client. Queue to exchange data between "client" and server.
'''
# This function intended to be used by the client
# Queue to exchange data between "client" and server.
retData = False
if not os.path.exists(self.queueDB):
conn = sqlite3.connect(self.queueDB)
@ -226,7 +292,7 @@ class Core:
retData = row
break
if retData != False:
c.execute('delete from commands where id = ?', (retData[3],))
c.execute('DELETE FROM commands WHERE id=?;', (retData[3],))
conn.commit()
conn.close()
@ -234,19 +300,23 @@ class Core:
def daemonQueueAdd(self, command, data=''):
'''
Add a command to the daemon queue, used by the communication daemon (communicator.py)
Add a command to the daemon queue, used by the communication daemon (communicator.py)
'''
# Intended to be used by the web server
date = math.floor(time.time())
conn = sqlite3.connect(self.queueDB)
c = conn.cursor()
t = (command, data, date)
c.execute('INSERT into commands (command, data, date) values (?, ?, ?)', t)
c.execute('INSERT INTO commands (command, data, date) VALUES(?, ?, ?)', t)
conn.commit()
conn.close()
return
def clearDaemonQueue(self):
'''clear the daemon queue (somewhat dangerousous)'''
'''
Clear the daemon queue (somewhat dangerous)
'''
conn = sqlite3.connect(self.queueDB)
c = conn.cursor()
try:
@ -256,58 +326,59 @@ class Core:
pass
conn.close()
def generateHMAC(self):
return
def listAdders(self, randomOrder=True, i2p=True):
'''
generate and return an HMAC key
Return a list of addresses
'''
key = base64.b64encode(os.urandom(32))
return key
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
if randomOrder:
addresses = c.execute('SELECT * FROM adders ORDER BY RANDOM();')
else:
addresses = c.execute('SELECT * FROM adders;')
addressList = []
for i in addresses:
addressList.append(i[0])
conn.close()
return addressList
def listPeers(self, randomOrder=True):
'''Return a list of peers
'''
Return a list of public keys (misleading function name)
randomOrder determines if the list should be in a random order
randomOrder determines if the list should be in a random order
'''
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
if randomOrder:
peers = c.execute('SELECT * FROM peers order by RANDOM();')
peers = c.execute('SELECT * FROM peers ORDER BY RANDOM();')
else:
peers = c.execute('SELECT * FROM peers;')
peerList = []
for i in peers:
peerList.append(i[0])
peerList.append(i[2])
conn.close()
return peerList
def processBlocks(self):
'''
Work with the block database and download any missing blocks
This is meant to be called from the communicator daemon on its timer.
'''
for i in self.getBlockList(True).split("\n"):
if i != "":
print('UNSAVED BLOCK:', i)
return
def getPeerInfo(self, peer, info):
'''
get info about a peer
Get info about a peer from their database entry
id text 0
name text, 1
pgpKey text, 2
hmacKey text, 3
blockDBHash text, 4
forwardKey text, 5
dateSeen not null, 7
bytesStored int, 8
trust int 9
id text 0
name text, 1
adders text, 2
forwardKey text, 3
dateSeen not null, 4
bytesStored int, 5
trust int 6
'''
# Lookup something about a peer from their database entry
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
command = (peer,)
infoNumbers = {'id': 0, 'name': 1, 'pgpKey': 2, 'hmacKey': 3, 'blockDBHash': 4, 'forwardKey': 5, 'dateSeen': 6, 'bytesStored': 7, 'trust': 8}
infoNumbers = {'id': 0, 'name': 1, 'adders': 2, 'forwardKey': 3, 'dateSeen': 4, 'bytesStored': 5, 'trust': 6}
info = infoNumbers[info]
iterCount = 0
retVal = ''
@ -319,28 +390,109 @@ class Core:
else:
iterCount += 1
conn.close()
return retVal
def setPeerInfo(self, peer, key, data):
'''update a peer for a key'''
'''
Update a peer for a key
'''
conn = sqlite3.connect(self.peerDB)
c = conn.cursor()
command = (data, peer)
# TODO: validate key on whitelist
c.execute('UPDATE peers SET ' + key + ' = ? where id=?', command)
if key not in ('id', 'name', 'pubkey', 'blockDBHash', 'forwardKey', 'dateSeen', 'bytesStored', 'trust'):
raise Exception("Got invalid database key when setting peer info")
c.execute('UPDATE peers SET ' + key + ' = ? WHERE id=?', command)
conn.commit()
conn.close()
return
def getAddressInfo(self, address, info):
'''
Get info about an address from its database entry
address text, 0
type int, 1
knownPeer text, 2
speed int, 3
success int, 4
DBHash text, 5
failure int 6
'''
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
command = (address,)
infoNumbers = {'address': 0, 'type': 1, 'knownPeer': 2, 'speed': 3, 'success': 4, 'DBHash': 5, 'failure': 6}
info = infoNumbers[info]
iterCount = 0
retVal = ''
for row in c.execute('SELECT * from adders where address=?;', command):
for i in row:
if iterCount == info:
retVal = i
break
else:
iterCount += 1
conn.close()
return retVal
def setAddressInfo(self, address, key, data):
'''
Update an address for a key
'''
conn = sqlite3.connect(self.addressDB)
c = conn.cursor()
command = (data, address)
# TODO: validate key on whitelist
if key not in ('address', 'type', 'knownPeer', 'speed', 'success', 'DBHash', 'failure'):
raise Exception("Got invalid database key when setting address info")
c.execute('UPDATE adders SET ' + key + ' = ? WHERE address=?', command)
conn.commit()
conn.close()
return
def getBlockList(self, unsaved=False):
'''get list of our blocks'''
'''
Get list of our blocks
'''
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
retData = ''
if unsaved:
execute = 'SELECT hash FROM hashes where dataSaved != 1;'
execute = 'SELECT hash FROM hashes WHERE dataSaved != 1;'
else:
execute = 'SELECT hash FROM hashes;'
for row in c.execute(execute):
for i in row:
retData += i + "\n"
return retData
def getBlocksByType(self, blockType):
'''
Returns a list of blocks by the type
'''
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
retData = ''
execute = 'SELECT hash FROM hashes WHERE dataType=?;'
args = (blockType,)
for row in c.execute(execute, args):
for i in row:
retData += i + "\n"
return retData.split('\n')
def setBlockType(self, hash, blockType):
'''
Sets the type of block
'''
conn = sqlite3.connect(self.blockDB)
c = conn.cursor()
c.execute("UPDATE hashes SET dataType='" + blockType + "' WHERE hash = '" + hash + "';")
conn.commit()
conn.close()
return

0
onionr/dev-enabled
View File

56
onionr/gui.py
View File

@ -13,4 +13,58 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
'''
from tkinter import *
import os, sqlite3, core
class OnionrGUI:
def __init__(self, myCore):
self.root = Tk()
self.myCore = myCore # onionr core
self.root.title("PyOnionr")
w = Label(self.root, text="Onionr", width=10)
w.config(font=("Sans-Serif", 22))
w.pack()
scrollbar = Scrollbar(self.root)
scrollbar.pack(side=RIGHT, fill=Y)
self.listedBlocks = []
idText = open('./data/hs/hostname', 'r').read()
idLabel = Label(self.root, text="ID: " + idText)
idLabel.pack(pady=5)
self.sendEntry = Entry(self.root)
sendBtn = Button(self.root, text='Send Message', command=self.sendMessage)
self.sendEntry.pack()
sendBtn.pack()
self.listbox = Listbox(self.root, yscrollcommand=scrollbar.set, height=15)
#listbox.insert(END, str(i))
self.listbox.pack(fill=BOTH)
scrollbar.config(command=self.listbox.yview)
self.root.after(2000, self.update)
self.root.mainloop()
def sendMessage(self):
messageToAdd = '-txt-' + self.sendEntry.get()
addedHash = self.myCore.setData(messageToAdd)
self.myCore.addToBlockDB(addedHash, selfInsert=True)
self.myCore.setBlockType(addedHash, 'txt')
self.sendEntry.delete(0, END)
def update(self):
for i in self.myCore.getBlocksByType('txt'):
if i.strip() == '' or i in self.listedBlocks:
continue
blockFile = open('./data/blocks/' + i + '.dat')
self.listbox.insert(END, str(blockFile.read().replace('-txt-', '')))
blockFile.close()
self.listedBlocks.append(i)
self.listbox.see(END)
blocksList = os.listdir('./data/blocks/') # dir is your directory path
number_blocks = len(blocksList)
self.root.after(10000, self.update)

91
onionr/logger.py
View File

@ -78,60 +78,82 @@ _type = OUTPUT_TO_CONSOLE | USE_ANSI # the default settings for logging
_level = LEVEL_DEBUG # the lowest level to log
_outputfile = './output.log' # the file to log to
'''
Set the settings for the logger using bitwise operators
'''
def set_settings(type):
'''
Set the settings for the logger using bitwise operators
'''
global _type
_type = type
'''
Get settings from the logger
'''
def get_settings():
'''
Get settings from the logger
'''
return _type
'''
Set the lowest log level to output
'''
def set_level(level):
'''
Set the lowest log level to output
'''
global _level
_level = level
'''
Get the lowest log level currently being outputted
'''
def get_level():
'''
Get the lowest log level currently being outputted
'''
return _level
'''
Outputs raw data to console without formatting
'''
def set_file(outputfile):
'''
Set the file to output to, if enabled
'''
global _outputfile
_outputfile = outputfile
def get_file():
'''
Get the file to output to
'''
return _outputfile
def raw(data):
'''
Outputs raw data to console without formatting
'''
if get_settings() & OUTPUT_TO_CONSOLE:
print(data)
if get_settings() & OUTPUT_TO_FILE:
with open(_outputfile, "a+") as f:
f.write(colors.filter(data) + '\n')
'''
Logs the data
prefix : The prefix to the output
data : The actual data to output
color : The color to output before the data
'''
def log(prefix, data, color = ''):
'''
Logs the data
prefix : The prefix to the output
data : The actual data to output
color : The color to output before the data
'''
output = colors.reset + str(color) + '[' + colors.bold + str(prefix) + colors.reset + str(color) + '] ' + str(data) + colors.reset
if not get_settings() & USE_ANSI:
output = colors.filter(output)
raw(output)
'''
Takes in input from the console, not stored in logs
message: The message to display before taking input
'''
def input(message = 'Enter input: '):
def readline(message = ''):
'''
Takes in input from the console, not stored in logs
message: The message to display before taking input
'''
color = colors.fg.green + colors.bold
output = colors.reset + str(color) + '... ' + colors.reset + str(message) + colors.reset
@ -139,14 +161,16 @@ def input(message = 'Enter input: '):
output = colors.filter(output)
sys.stdout.write(output)
return raw_input()
'''
Displays an "Are you sure" message, returns True for Y and False for N
message: The confirmation message, use %s for (y/n)
default: which to prefer-- y or n
'''
return input()
def confirm(default = 'y', message = 'Are you sure %s? '):
'''
Displays an "Are you sure" message, returns True for Y and False for N
message: The confirmation message, use %s for (y/n)
default: which to prefer-- y or n
'''
color = colors.fg.green + colors.bold
default = default.lower()
@ -163,7 +187,8 @@ def confirm(default = 'y', message = 'Are you sure %s? '):
output = colors.filter(output)
sys.stdout.write(output.replace('%s', confirm))
inp = raw_input().lower()
inp = input().lower()
if 'y' in inp:
return True

69
onionr/netcontroller.py
View File

@ -17,11 +17,14 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import subprocess, os, random, sys, logger, time, signal
class NetController:
'''NetController
This class handles hidden service setup on Tor and I2P
'''
This class handles hidden service setup on Tor and I2P
'''
def __init__(self, hsPort):
self.torConfigLocation = 'data/torrc'
self.readyState = False
@ -30,15 +33,20 @@ class NetController:
self._torInstnace = ''
self.myID = ''
'''
if os.path.exists(self.torConfigLocation):
torrc = open(self.torConfigLocation, 'r')
if not str(self.hsPort) in torrc.read():
os.remove(self.torConfigLocation)
torrc.close()
if os.path.exists(self.torConfigLocation):
torrc = open(self.torConfigLocation, 'r')
if not str(self.hsPort) in torrc.read():
os.remove(self.torConfigLocation)
torrc.close()
'''
return
def generateTorrc(self):
'''generate a torrc file for our tor instance'''
'''
Generate a torrc file for our tor instance
'''
if os.path.exists(self.torConfigLocation):
os.remove(self.torConfigLocation)
torrcData = '''SocksPort ''' + str(self.socksPort) + '''
@ -48,50 +56,83 @@ HiddenServicePort 80 127.0.0.1:''' + str(self.hsPort) + '''
torrc = open(self.torConfigLocation, 'w')
torrc.write(torrcData)
torrc.close()
return
def startTor(self):
'''Start Tor with onion service on port 80 & socks proxy on random port
'''
Start Tor with onion service on port 80 & socks proxy on random port
'''
self.generateTorrc()
if os.path.exists('./tor'):
torBinary = './tor'
elif os.path.exists('/usr/bin/tor'):
torBinary = '/usr/bin/tor'
else:
torBinary = 'tor'
try:
tor = subprocess.Popen([torBinary, '-f', self.torConfigLocation], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
except FileNotFoundError:
logger.fatal("Tor was not found in your path or the Onionr directory. Please install Tor and try again.")
sys.exit(1)
else:
# Test Tor Version
torVersion = subprocess.Popen([torBinary, '--version'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for line in iter(torVersion.stdout.readline, b''):
if 'Tor 0.2.' in line.decode():
logger.warn("Running 0.2.x Tor series, no support for v3 onion peers")
break
torVersion.kill()
# wait for tor to get to 100% bootstrap
for line in iter(tor.stdout.readline, b''):
if 'Bootstrapped 100%: Done' in line.decode():
break
elif 'Opening Socks listener' in line.decode():
logger.debug(line.decode())
logger.debug(line.decode().replace('\n', ''))
else:
logger.fatal('Failed to start Tor. Try killing any other Tor processes owned by this user.')
return False
logger.info('Finished starting Tor')
self.readyState = True
myID = open('data/hs/hostname', 'r')
self.myID = myID.read()
self.myID = myID.read().replace('\n', '')
myID.close()
torPidFile = open('data/torPid.txt', 'w')
torPidFile.write(str(tor.pid))
torPidFile.close()
return True
def killTor(self):
'''properly kill tor based on pid saved to file'''
'''
Properly kill tor based on pid saved to file
'''
try:
pid = open('data/torPid.txt', 'r')
pidN = pid.read()
pid.close()
except FileNotFoundError:
return
try:
int(pidN)
except:
return
os.kill(int(pidN), signal.SIGTERM)
os.remove('data/torPid.txt')
try:
os.kill(int(pidN), signal.SIGTERM)
os.remove('data/torPid.txt')
except ProcessLookupError:
pass
except FileNotFoundError:
pass
return

420
onionr/onionr.py
View File

@ -20,8 +20,8 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import sys, os, configparser, base64, random, getpass, shutil, subprocess, requests, time, logger
import gui, api, core
import sys, os, base64, random, getpass, shutil, subprocess, requests, time, platform
import api, core, gui, config, logger, onionrplugins as plugins
from onionrutils import OnionrUtils
from netcontroller import NetController
@ -30,20 +30,44 @@ try:
except ImportError:
raise Exception("You need the PySocks module (for use with socks5 proxy to use Tor)")
class Onionr:
def __init__(self):
'''Main Onionr class. This is for the CLI program, and does not handle much of the logic.
In general, external programs and plugins should not use this class.
ONIONR_TAGLINE = 'Anonymous P2P Platform - GPLv3 - onionr.voidnet.tech'
ONIONR_VERSION = '0.0.0' # for debugging and stuff
API_VERSION = '1' # increments of 1; only change when something fundemental about how the API works changes. This way other nodes knows how to communicate without learning too much information about you.
class Onionr:
cmds = {}
cmdhelp = {}
def __init__(self):
'''
Main Onionr class. This is for the CLI program, and does not handle much of the logic.
In general, external programs and plugins should not use this class.
'''
try:
os.chdir(sys.path[0])
except FileNotFoundError:
pass
if os.path.exists('dev-enabled'):
# Load global configuration data
exists = os.path.exists(config.get_config_file())
config.set_config({'devmode': True, 'log': {'file': {'output': True, 'path': 'data/output.log'}, 'console': {'output': True, 'color': True}}}) # this is the default config, it will be overwritten if a config file already exists. Else, it saves it
config.reload() # this will read the configuration file into memory
settings = 0b000
if config.get('log', {'console': {'color': True}})['console']['color']:
settings = settings | logger.USE_ANSI
if config.get('log', {'console': {'output': True}})['console']['output']:
settings = settings | logger.OUTPUT_TO_CONSOLE
if config.get('log', {'file': {'output': True}})['file']['output']:
settings = settings | logger.OUTPUT_TO_FILE
logger.set_file(config.get('log', {'file': {'path': 'data/output.log'}})['file']['path'])
logger.set_settings(settings)
if config.get('devmode', True):
self._developmentMode = True
logger.set_level(logger.LEVEL_DEBUG)
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
else:
self._developmentMode = False
logger.set_level(logger.LEVEL_INFO)
@ -51,7 +75,7 @@ class Onionr:
self.onionrCore = core.Core()
self.onionrUtils = OnionrUtils(self.onionrCore)
# Get configuration and Handle commands
# Handle commands
self.debug = False # Whole application debugging
@ -69,15 +93,15 @@ class Onionr:
os.mkdir('data/')
os.mkdir('data/blocks/')
if not os.path.exists('data/peers.db'):
if not os.path.exists(self.onionrCore.peerDB):
self.onionrCore.createPeerDB()
pass
if not os.path.exists(self.onionrCore.addressDB):
self.onionrCore.createAddressDB()
# Get configuration
self.config = configparser.ConfigParser()
if os.path.exists('data/config.ini'):
self.config.read('data/config.ini')
else:
if not exists:
# Generate default config
# Hostname should only be set if different from 127.x.x.x. Important for DNS rebinding attack prevention.
if self.debug:
@ -87,78 +111,364 @@ class Onionr:
randomPort = random.randint(1024, 65535)
if self.onionrUtils.checkPort(randomPort):
break
self.config['CLIENT'] = {'CLIENT HMAC': base64.b64encode(os.urandom(32)).decode('utf-8'), 'PORT': randomPort, 'API VERSION': '0.0.0'}
with open('data/config.ini', 'w') as configfile:
self.config.write(configfile)
config.set('client', {'participate': 'true', 'client_hmac': base64.b64encode(os.urandom(32)).decode('utf-8'), 'port': randomPort, 'api_version': API_VERSION}, True)
self.cmds = {
'': self.showHelpSuggestion,
'help': self.showHelp,
'version': self.version,
'config': self.configure,
'start': self.start,
'stop': self.killDaemon,
'stats': self.showStats,
'enable-plugin': self.enablePlugin,
'enplugin': self.enablePlugin,
'enableplugin': self.enablePlugin,
'enmod': self.enablePlugin,
'disable-plugin': self.disablePlugin,
'displugin': self.disablePlugin,
'disableplugin': self.disablePlugin,
'dismod': self.disablePlugin,
'reload-plugin': self.reloadPlugin,
'reloadplugin': self.reloadPlugin,
'reload-plugins': self.reloadPlugin,
'reloadplugins': self.reloadPlugin,
'listpeers': self.listPeers,
'list-peers': self.listPeers,
'addmsg': self.addMessage,
'addmessage': self.addMessage,
'add-msg': self.addMessage,
'add-message': self.addMessage,
'pm': self.sendEncrypt,
'gui': self.openGUI,
'addpeer': self.addPeer,
'add-peer': self.addPeer,
'add-address': self.addAddress,
'addaddress': self.addAddress,
'connect': self.addAddress
}
self.cmdhelp = {
'help': 'Displays this Onionr help menu',
'version': 'Displays the Onionr version',
'config': 'Configures something and adds it to the file',
'start': 'Starts the Onionr daemon',
'stop': 'Stops the Onionr daemon',
'stats': 'Displays node statistics',
'enable-plugin': 'Enables and starts a plugin',
'disable-plugin': 'Disables and stops a plugin',
'reload-plugin': 'Reloads a plugin',
'list-peers': 'Displays a list of peers',
'add-peer': 'Adds a peer (?)',
'add-msg': 'Broadcasts a message to the Onionr network',
'pm': 'Adds a private message to block',
'gui': 'Opens a graphical interface for Onionr'
}
command = ''
try:
command = sys.argv[1].lower()
except IndexError:
command = ''
finally:
if command == 'start':
if os.path.exists('.onionr-lock'):
logger.fatal('Cannot start. Daemon is already running, or it did not exit cleanly.\n(if you are sure that there is not a daemon running, delete .onionr-lock & try again).')
else:
if not self.debug and not self._developmentMode:
lockFile = open('.onionr-lock', 'w')
lockFile.write('')
lockFile.close()
self.daemon()
if not self.debug and not self._developmentMode:
os.remove('.onionr-lock')
elif command == 'stop':
self.killDaemon()
elif command in ('addmsg', 'addmessage'):
while True:
messageToAdd = input('Broadcast message to network: ')
if len(messageToAdd) >= 1:
break
addedHash = self.onionrCore.setData(messageToAdd)
self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
elif command == 'stats':
self.showStats()
elif command == 'help' or command == '--help':
self.showHelp()
elif command == '':
logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
else:
logger.error('Invalid command.')
self.execute(command)
if not self._developmentMode:
encryptionPassword = self.onionrUtils.getPassword('Enter password to encrypt directory: ')
self.onionrCore.dataDirEncrypt(encryptionPassword)
shutil.rmtree('data/')
return
def daemon(self):
''' Start the Onionr communication daemon
'''
THIS SECTION HANDLES THE COMMANDS
'''
def getCommands(self):
return self.cmds
def getHelp(self):
return self.cmdhelp
def addCommand(self, command, function):
cmds[str(command).lower()] = function
def addHelp(self, command, description):
cmdhelp[str(command).lower()] = str(description)
def configure(self):
'''
Displays something from the configuration file, or sets it
'''
if len(sys.argv) >= 4:
config.reload()
config.set(sys.argv[2], sys.argv[3], True)
logger.debug('Configuration file updated.')
elif len(sys.argv) >= 3:
config.reload()
logger.info(logger.colors.bold + sys.argv[2] + ': ' + logger.colors.reset + str(config.get(sys.argv[2], logger.colors.fg.red + 'Not set.')))
else:
logger.info(logger.colors.bold + 'Get a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key>')
logger.info(logger.colors.bold + 'Set a value: ' + logger.colors.reset + sys.argv[0] + ' ' + sys.argv[1] + ' <key> <value>')
def execute(self, argument):
'''
Executes a command
'''
argument = argument[argument.startswith('--') and len('--'):] # remove -- if it starts with it
# define commands
commands = self.getCommands()
command = commands.get(argument, self.notFound)
command()
'''
THIS SECTION DEFINES THE COMMANDS
'''
def version(self, verbosity=5):
'''
Displays the Onionr version
'''
logger.info('Onionr ' + ONIONR_VERSION + ' (' + platform.machine() + ') - API v' + API_VERSION)
if verbosity >= 1:
logger.info(ONIONR_TAGLINE)
if verbosity >= 2:
logger.info('Running on ' + platform.platform() + ' ' + platform.release())
def sendEncrypt(self):
'''
Create a private message and send it
'''
while True:
try:
peer = logger.readline('Peer to send to: ')
except KeyboardInterrupt:
break
else:
if self.onionrUtils.validateID(peer):
break
else:
logger.error('Invalid peer ID')
else:
try:
message = logger.readline("Enter a message: ")
except KeyboardInterrupt:
pass
else:
logger.info("Sending message to " + peer)
self.onionrUtils.sendPM(peer, message)
def openGUI(self):
'''
Opens a graphical interface for Onionr
'''
gui.OnionrGUI(self.onionrCore)
def listPeers(self):
'''
Displays a list of peers (?)
'''
logger.info('Peer list:\n')
for i in self.onionrCore.listPeers():
logger.info(i)
def addPeer(self):
'''
Adds a peer (?)
'''
try:
newPeer = sys.argv[2]
except:
pass
else:
logger.info("Adding peer: " + logger.colors.underline + newPeer)
self.onionrCore.addPeer(newPeer)
return
def addAddress(self):
'''Adds a Onionr node address'''
try:
newAddress = sys.argv[2]
except:
pass
else:
logger.info("Adding address: " + logger.colors.underline + newAddress)
if self.onionrCore.addAddress(newAddress):
logger.info("Successfully added address")
else:
logger.warn("Unable to add address")
return
def addMessage(self):
'''
Broadcasts a message to the Onionr network
'''
while True:
messageToAdd = '-txt-' + logger.readline('Broadcast message to network: ')
if len(messageToAdd) >= 1:
break
addedHash = self.onionrCore.setData(messageToAdd)
self.onionrCore.addToBlockDB(addedHash, selfInsert=True)
self.onionrCore.setBlockType(addedHash, 'txt')
return
def enablePlugin(self):
'''
Enables and starts the given plugin
'''
if len(sys.argv) >= 3:
plugin_name = sys.argv[2]
logger.info('Enabling plugin \"' + plugin_name + '\"...')
plugins.enable(plugin_name)
else:
logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
return
def disablePlugin(self):
'''
Disables and stops the given plugin
'''
if len(sys.argv) >= 3:
plugin_name = sys.argv[2]
logger.info('Disabling plugin \"' + plugin_name + '\"...')
plugins.disable(plugin_name)
else:
logger.info(sys.argv[0] + ' ' + sys.argv[1] + ' <plugin>')
return
def reloadPlugin(self):
'''
Reloads (stops and starts) all plugins, or the given plugin
'''
if len(sys.argv) >= 3:
plugin_name = sys.argv[2]
logger.info('Reloading plugin \"' + plugin_name + '\"...')
plugins.stop(plugin_name)
plugins.start(plugin_name)
else:
logger.info('Reloading all plugins...')
plugins.reload()
return
def notFound(self):
'''
Displays a "command not found" message
'''
logger.error('Command not found.')
def showHelpSuggestion(self):
'''
Displays a message suggesting help
'''
logger.info('Do ' + logger.colors.bold + sys.argv[0] + ' --help' + logger.colors.reset + logger.colors.fg.green + ' for Onionr help.')
def start(self):
'''
Starts the Onionr daemon
'''
if os.path.exists('.onionr-lock'):
logger.fatal('Cannot start. Daemon is already running, or it did not exit cleanly.\n(if you are sure that there is not a daemon running, delete .onionr-lock & try again).')
else:
if not self.debug and not self._developmentMode:
lockFile = open('.onionr-lock', 'w')
lockFile.write('')
lockFile.close()
self.daemon()
if not self.debug and not self._developmentMode:
os.remove('.onionr-lock')
def daemon(self):
'''
Starts the Onionr communication daemon
'''
if not os.environ.get("WERKZEUG_RUN_MAIN") == "true":
net = NetController(self.config['CLIENT']['PORT'])
if self._developmentMode:
logger.warn('DEVELOPMENT MODE ENABLED (THIS IS LESS SECURE!)')
net = NetController(config.get('client')['port'])
logger.info('Tor is starting...')
if not net.startTor():
sys.exit(1)
logger.info('Started Tor .onion service: ' + logger.colors.underline + net.myID)
logger.info('Our Public key: ' + self.onionrCore._crypto.pubKey)
time.sleep(1)
subprocess.Popen(["./communicator.py", "run", str(net.socksPort)])
logger.debug('Started communicator')
api.API(self.config, self.debug)
api.API(self.debug)
return
def killDaemon(self):
'''Shutdown the Onionr Daemon'''
'''
Shutdown the Onionr daemon
'''
logger.warn('Killing the running daemon')
net = NetController(self.config['CLIENT']['PORT'])
net = NetController(config.get('client')['port'])
try:
self.onionrUtils.localCommand('shutdown')
except requests.exceptions.ConnectionError:
pass
self.onionrCore.daemonQueueAdd('shutdown')
net.killTor()
return
def showStats(self):
'''Display statistics and exit'''
'''
Displays statistics and exits
'''
return
def showHelp(self):
'''Show help for Onionr'''
def showHelp(self, command = None):
'''
Show help for Onionr
'''
helpmenu = self.getHelp()
if command is None and len(sys.argv) >= 3:
for cmd in sys.argv[2:]:
self.showHelp(cmd)
elif not command is None:
if command.lower() in helpmenu:
logger.info(logger.colors.bold + command + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset + helpmenu[command.lower()])
else:
logger.warn(logger.colors.bold + command + logger.colors.reset + logger.colors.fg.blue + ' : ' + logger.colors.reset + 'No help menu entry was found')
else:
self.version(0)
for command, helpmessage in helpmenu.items():
self.showHelp(command)
return
Onionr()
Onionr()

97
onionr/onionrcrypto.py Normal file
View File

@ -0,0 +1,97 @@
'''
Onionr - P2P Microblogging Platform & Social network
This file handles Onionr's cryptography.
'''
'''
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import nacl.signing, nacl.encoding, nacl.public, os
class OnionrCrypto:
def __init__(self, coreInstance):
self._core = coreInstance
self._keyFile = 'data/keys.txt'
self.pubKey = None
self.privKey = None
# Load our own pub/priv Ed25519 keys, gen & save them if they don't exist
if os.path.exists(self._keyFile):
with open('data/keys.txt', 'r') as keys:
keys = keys.read().split(',')
self.pubKey = keys[0]
self.privKey = keys[1]
else:
keys = self.generatePubKey()
self.pubKey = keys[0]
self.privKey = keys[1]
with open(self._keyFile, 'w') as keyfile:
keyfile.write(self.pubKey + ',' + self.privKey)
return
def edVerify(self, data, key):
'''Verify signed data (combined in nacl) to an ed25519 key'''
key = nacl.signing.VerifyKey(key=key, encoder=nacl.encoding.Base32Encoder)
retData = ''
if encodeResult:
retData = key.verify(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
else:
retData = key.verify(data.encode())
return retData
def edSign(self, data, key, encodeResult=False):
'''Ed25519 sign data'''
key = nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
retData = ''
if encodeResult:
retData = key.sign(data.encode(), encoder=nacl.encoding.Base64Encoder) # .encode() is not the same as nacl.encoding
else:
retData = key.sign(data.encode())
return retData
def pubKeyEncrypt(self, data, pubkey, anonymous=False):
'''Encrypt to a public key (Curve25519, taken from base32 Ed25519 pubkey)'''
retVal = ''
if self.privKey != None and not anonymous:
ownKey = nacl.signing.SigningKey(seed=self.privKey, encoder=nacl.encoding.Base32Encoder())
key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
ourBox = nacl.public.Box(ownKey, key)
retVal = ourBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
elif anonymous:
key = nacl.signing.VerifyKey(key=pubkey, encoder=nacl.encoding.Base32Encoder).to_curve25519_public_key()
anonBox = nacl.public.SealedBox(key)
retVal = anonBox.encrypt(data.encode(), encoder=nacl.encoding.RawEncoder)
return retVal
def pubKeyDecrypt(self, data, peer):
'''pubkey decrypt (Curve25519, taken from Ed25519 pubkey)'''
return
def symmetricPeerEncrypt(self, data):
'''Salsa20 encrypt data to peer (with mac)'''
return
def symmetricPeerDecrypt(self, data, peer):
'''Salsa20 decrypt data from peer (with mac)'''
return
def generateSymmetric(self, data, peer):
'''Generate symmetric key'''
return
def generatePubKey(self):
'''Generate a Ed25519 public key pair, return tuple of base64encoded pubkey, privkey'''
private_key = nacl.signing.SigningKey.generate()
public_key = private_key.verify_key.encode(encoder=nacl.encoding.Base32Encoder())
return (public_key.decode(), private_key.encode(encoder=nacl.encoding.Base32Encoder()).decode())

53
onionr/onionrevents.py Normal file
View File

@ -0,0 +1,53 @@
'''
Onionr - P2P Microblogging Platform & Social network
This file deals with configuration management.
'''
'''
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import config, logger, onionrplugins as plugins
def event(event_name, data = None, onionr = None):
'''
Calls an event on all plugins (if defined)
'''
for plugin in plugins.get_enabled_plugins():
try:
call(plugins.get_plugin(plugin), event_name, data, onionr)
except:
logger.warn('Event \"' + event_name + '\" failed for plugin \"' + plugin + '\".')
def call(plugin, event_name, data = None, onionr = None):
'''
Calls an event on a plugin if one is defined
'''
if not plugin is None:
try:
attribute = 'on_' + str(event_name).lower()
# TODO: Use multithreading perhaps?
if hasattr(plugin, attribute):
logger.debug('Calling event ' + str(event_name))
getattr(plugin, attribute)(onionr, data)
return True
except:
logger.warn('Failed to call event ' + str(event_name) + ' on module.')
return False
else:
return True

234
onionr/onionrplugins.py Normal file
View File

@ -0,0 +1,234 @@
'''
Onionr - P2P Microblogging Platform & Social network
This file deals with management of modules/plugins.
'''
'''
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import os, re, importlib, config, logger
import onionrevents as events
_pluginsfolder = 'data/plugins/'
_instances = dict()
def reload(stop_event = True):
'''
Reloads all the plugins
'''
check()
try:
enabled_plugins = get_enabled_plugins()
if stop_event is True:
logger.debug('Reloading all plugins...')
else:
logger.debug('Loading all plugins...')
if stop_event is True:
for plugin in enabled_plugins:
stop(plugin)
for plugin in enabled_plugins:
start(plugin)
return True
except:
logger.error('Failed to reload plugins.')
return False
def enable(name, start_event = True):
'''
Enables a plugin
'''
check()
if exists(name):
enabled_plugins = get_enabled_plugins()
enabled_plugins.append(name)
config_plugins = config.get('plugins')
config_plugins['enabled'] = enabled_plugins
config.set('plugins', config_plugins, True)
events.call(get_plugin(name), 'enable')
if start_event is True:
start(name)
return True
else:
logger.error('Failed to enable plugin \"' + name + '\", disabling plugin.')
disable(name)
return False
def disable(name, stop_event = True):
'''
Disables a plugin
'''
check()
if is_enabled(name):
enabled_plugins = get_enabled_plugins()
enabled_plugins.remove(name)
config_plugins = config.get('plugins')
config_plugins['enabled'] = enabled_plugins
config.set('plugins', config_plugins, True)
if exists(name):
events.call(get_plugin(name), 'disable')
if stop_event is True:
stop(name)
def start(name):
'''
Starts the plugin
'''
check()
if exists(name):
try:
plugin = get_plugin(name)
if plugin is None:
raise Exception('Failed to import module.')
else:
events.call(plugin, 'start')
return plugin
except:
logger.error('Failed to start module \"' + name + '\".')
else:
logger.error('Failed to start nonexistant module \"' + name + '\".')
return None
def stop(name):
'''
Stops the plugin
'''
check()
if exists(name):
try:
plugin = get_plugin(name)
if plugin is None:
raise Exception('Failed to import module.')
else:
events.call(plugin, 'stop')
return plugin
except:
logger.error('Failed to stop module \"' + name + '\".')
else:
logger.error('Failed to stop nonexistant module \"' + name + '\".')
return None
def get_plugin(name):
'''
Returns the instance of a module
'''
check()
if str(name).lower() in _instances:
return _instances[str(name).lower()]
else:
_instances[str(name).lower()] = importlib.import_module(get_plugins_folder(name, False).replace('/', '.') + 'main')
return get_plugin(name)
def get_plugins():
'''
Returns a list of plugins (deprecated)
'''
return _instances
def exists(name):
'''
Return value indicates whether or not the plugin exists
'''
return os.path.isdir(get_plugins_folder(str(name).lower()))
def get_enabled_plugins():
'''
Returns a list of the enabled plugins
'''
check()
config.reload()
return config.get('plugins')['enabled']
def is_enabled(name):
'''
Return value indicates whether or not the plugin is enabled
'''
return name in get_enabled_plugins()
def get_plugins_folder(name = None, absolute = True):
'''
Returns the path to the plugins folder
'''
path = ''
if name is None:
path = _pluginsfolder
else:
# only allow alphanumeric characters
path = _pluginsfolder + re.sub('[^0-9a-zA-Z]+', '', str(name).lower()) + '/'
if absolute is True:
path = os.path.abspath(path)
return path
def check():
'''
Checks to make sure files exist
'''
config.reload()
if not config.is_set('plugins'):
logger.debug('Generating plugin config data...')
config.set('plugins', {'enabled': []}, True)
if not os.path.exists(os.path.dirname(get_plugins_folder())):
logger.debug('Generating plugin data folder...')
os.makedirs(os.path.dirname(get_plugins_folder()))
if not exists('test'):
os.makedirs(get_plugins_folder('test'))
with open(get_plugins_folder('test') + '/main.py', 'a') as main:
main.write("print('Running')\n\ndef on_test(onionr = None, data = None):\n print('received test event!')\n return True\n\ndef on_start(onionr = None, data = None):\n print('start event called')\n\ndef on_stop(onionr = None, data = None):\n print('stop event called')\n\ndef on_enable(onionr = None, data = None):\n print('enable event called')\n\ndef on_disable(onionr = None, data = None):\n print('disable event called')\n")
enable('test')
return

86
onionr/onionrproofs.py Normal file
View File

@ -0,0 +1,86 @@
'''
Onionr - P2P Microblogging Platform & Social network
Proof of work module
'''
'''
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import nacl.encoding, nacl.hash, nacl.utils, time, math, threading, binascii, logger
import btc
class POW:
def pow(self, reporting=False):
startTime = math.floor(time.time())
self.hashing = True
self.reporting = reporting
iFound = False # if current thread is the one that found the answer
answer = ''
heartbeat = 200000
hbCount = 0
blockCheck = 300000 # How often the hasher should check if the bitcoin block is updated (slows hashing but prevents less wasted work)
blockCheckCount = 0
block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
while self.hashing:
if blockCheckCount == blockCheck:
if self.reporting:
logger.debug('Refreshing Bitcoin block')
block = ''#self.bitcoinNode.getBlockHash(self.bitcoinNode.getLastBlockHeight())
blockCheckCount = 0
blockCheckCount += 1
hbCount += 1
token = nacl.hash.blake2b(nacl.utils.random() + block.encode()).decode()
if self.mainHash[0:self.difficulty] == token[0:self.difficulty]:
self.hashing = False
iFound = True
break
if iFound:
endTime = math.floor(time.time())
if self.reporting:
logger.info('Found token ' + token)
logger.info('took ' + str(endTime - startTime))
self.result = token
def __init__(self, difficulty, bitcoinNode):
self.foundHash = False
self.difficulty = difficulty
logger.debug('Computing difficulty of ' + str(self.difficulty))
self.mainHash = nacl.hash.blake2b(nacl.utils.random()).decode()
self.puzzle = self.mainHash[0:self.difficulty]
self.bitcoinNode = bitcoinNode
logger.debug('trying to find ' + str(self.mainHash))
tOne = threading.Thread(name='one', target=self.pow, args=(True,))
tTwo = threading.Thread(name='two', target=self.pow)
tThree = threading.Thread(name='three', target=self.pow)
tOne.start()
tTwo.start()
tThree.start()
return
def shutdown(self):
self.hashing = False
self.puzzle = ''
def changeDifficulty(self, newDiff):
self.difficulty = newDiff
def getResult(self):
'''Returns the result then sets to false, useful to automatically clear the result'''
try:
retVal = self.result
except AttributeError:
retVal = False
self.result = False
return retVal

144
onionr/onionrutils.py
View File

@ -18,29 +18,78 @@
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
# Misc functions that do not fit in the main api, but are useful
import getpass, sys, requests, configparser, os, socket, gnupg, hashlib, logger
import getpass, sys, requests, os, socket, hashlib, logger, sqlite3, config
import nacl.signing, nacl.encoding
if sys.version_info < (3, 6):
try:
import sha3
except ModuleNotFoundError:
logger.fatal('On Python 3 versions prior to 3.6.x, you need the sha3 module')
sys.exit(1)
class OnionrUtils:
'''Various useful functions'''
'''
Various useful function
'''
def __init__(self, coreInstance):
self.fingerprintFile = 'data/own-fingerprint.txt'
self._core = coreInstance
return
def sendPM(self, user, message):
'''High level function to encrypt a message to a peer and insert it as a block'''
return
def incrementAddressSuccess(self, address):
'''Increase the recorded sucesses for an address'''
increment = self._core.getAddressInfo(address, 'success') + 1
self._core.setAddressInfo(address, 'success', increment)
return
def decrementAddressSuccess(self, address):
'''Decrease the recorded sucesses for an address'''
increment = self._core.getAddressInfo(address, 'success') - 1
self._core.setAddressInfo(address, 'success', increment)
return
def mergeKeys(self, newKeyList):
'''Merge ed25519 key list to our database'''
retVal = False
if newKeyList != False:
for key in newKeyList:
if not key in self._core.listPeers(randomOrder=False):
if self._core.addPeer(key):
retVal = True
return retVal
def mergeAdders(self, newAdderList):
'''Merge peer adders list to our database'''
retVal = False
if newAdderList != False:
for adder in newAdderList:
if not adder in self._core.listAdders(randomOrder=False):
if self._core.addAddress(adder):
retVal = True
return retVal
def localCommand(self, command):
'''Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.'''
config = configparser.ConfigParser()
if os.path.exists('data/config.ini'):
config.read('data/config.ini')
else:
return
requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config['CLIENT']['PORT']) + '/client/?action=' + command + '&token=' + config['CLIENT']['CLIENT HMAC'])
'''
Send a command to the local http API server, securely. Intended for local clients, DO NOT USE for remote peers.
'''
config.reload()
# TODO: URL encode parameters, just as an extra measure. May not be needed, but should be added regardless.
requests.get('http://' + open('data/host.txt', 'r').read() + ':' + str(config.get('client')['port']) + '/client/?action=' + command + '&token=' + str(config.get('client')['client_hmac']))
return
def getPassword(self, message='Enter password: ', confirm = True):
'''Get a password without showing the users typing and confirm the input'''
'''
Get a password without showing the users typing and confirm the input
'''
# Get a password safely with confirmation and return it
while True:
print(message)
@ -50,14 +99,18 @@ class OnionrUtils:
pass2 = getpass.getpass()
if pass1 != pass2:
logger.error("Passwords do not match.")
input()
logger.readline()
else:
break
else:
break
return pass1
def checkPort(self, port, host = ''):
'''Checks if a port is available, returns bool'''
def checkPort(self, port, host=''):
'''
Checks if a port is available, returns bool
'''
# inspired by https://www.reddit.com/r/learnpython/comments/2i4qrj/how_to_write_a_python_script_that_checks_to_see/ckzarux/
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
retVal = False
@ -68,37 +121,58 @@ class OnionrUtils:
retVal = True
finally:
sock.close()
return retVal
def checkIsIP(self, ip):
'''Check if a string is a valid ipv4 address'''
'''
Check if a string is a valid IPv4 address
'''
try:
socket.inet_aton(ip)
except:
return False
else:
return True
def exportMyPubkey(self):
'''Export our PGP key if it exists'''
if not os.path.exists(self.fingerprintFile):
raise Exception("No fingerprint found, cannot export our PGP key.")
gpg = gnupg.GPG(homedir='./data/pgp/')
with open(self.fingerprintFile,'r') as f:
fingerprint = f.read()
ascii_armored_public_keys = gpg.export_keys(fingerprint)
return ascii_armored_public_keys
def getBlockDBHash(self):
'''Return a sha3_256 hash of the blocks DB'''
'''
Return a sha3_256 hash of the blocks DB
'''
with open(self._core.blockDB, 'rb') as data:
data = data.read()
hasher = hashlib.sha3_256()
hasher.update(data)
dataHash = hasher.hexdigest()
return dataHash
def hasBlock(self, hash):
'''
Check for new block in the list
'''
conn = sqlite3.connect(self._core.blockDB)
c = conn.cursor()
if not self.validateHash(hash):
raise Exception("Invalid hash")
for result in c.execute("SELECT COUNT() FROM hashes where hash='" + hash + "'"):
if result[0] >= 1:
conn.commit()
conn.close()
return True
else:
conn.commit()
conn.close()
return False
def validateHash(self, data, length=64):
'''Validate if a string is a valid hex formatted hash'''
'''
Validate if a string is a valid hex formatted hash
'''
retVal = True
if data == False or data == True:
return False
data = data.strip()
if len(data) != length:
retVal = False
else:
@ -106,9 +180,25 @@ class OnionrUtils:
int(data, 16)
except ValueError:
retVal = False
return retVal
def validatePubKey(self, key):
'''Validate if a string is a valid base32 encoded Ed25519 key'''
retVal = False
try:
nacl.signing.SigningKey(seed=key, encoder=nacl.encoding.Base32Encoder)
except nacl.exceptions.ValueError:
pass
else:
retVal = True
return retVal
def validateID(self, id):
'''validate if a user ID is a valid tor or i2p hidden service'''
'''
Validate if an address is a valid tor or i2p hidden service
'''
idLength = len(id)
retVal = True
idNoDomain = ''
@ -146,5 +236,5 @@ class OnionrUtils:
retVal = False
if not idNoDomain.isalnum():
retVal = False
return retVal
return retVal

155
onionr/tests.py
View File

@ -14,7 +14,7 @@
You should have received a copy of the GNU General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger
import unittest, sys, os, base64, tarfile, shutil, simplecrypt, logger, btc
class OnionrTests(unittest.TestCase):
def testPython3(self):
@ -23,18 +23,21 @@ class OnionrTests(unittest.TestCase):
self.assertTrue(False)
else:
self.assertTrue(True)
def testNone(self):
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running simple program run test...')
# Test just running ./onionr with no arguments
blank = os.system('./onionr.py')
blank = os.system('./onionr.py --version')
if blank != 0:
self.assertTrue(False)
else:
self.assertTrue(True)
def testPeer_a_DBCreation(self):
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running peer db creation test...')
if os.path.exists('data/peers.db'):
os.remove('data/peers.db')
import core
@ -44,22 +47,27 @@ class OnionrTests(unittest.TestCase):
self.assertTrue(True)
else:
self.assertTrue(False)
def testPeer_b_addPeerToDB(self):
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running peer db insertion test...')
import core
myCore = core.Core()
if not os.path.exists('data/peers.db'):
myCore.createPeerDB()
if myCore.addPeer('facebookcorewwwi.onion') and not myCore.addPeer('invalidpeer.onion'):
if myCore.addPeer('6M5MXL237OK57ITHVYN5WGHANPGOMKS5C3PJLHBBNKFFJQOIDOJA====') and not myCore.addPeer('NFXHMYLMNFSAU==='):
self.assertTrue(True)
else:
self.assertTrue(False)
def testData_b_Encrypt(self):
self.assertTrue(True)
return
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running data dir encrypt test...')
import core
myCore = core.Core()
myCore.dataDirEncrypt('password')
@ -67,11 +75,14 @@ class OnionrTests(unittest.TestCase):
self.assertTrue(True)
else:
self.assertTrue(False)
def testData_a_Decrypt(self):
self.assertTrue(True)
return
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running data dir decrypt test...')
import core
myCore = core.Core()
myCore.dataDirDecrypt('password')
@ -79,34 +90,82 @@ class OnionrTests(unittest.TestCase):
self.assertTrue(True)
else:
self.assertTrue(False)
def testPGPGen(self):
logger.debug('--------------------------')
logger.info('Running PGP key generation test...')
if os.path.exists('data/pgp/'):
self.assertTrue(True)
else:
import core, netcontroller
myCore = core.Core()
net = netcontroller.NetController(1337)
net.startTor()
torID = open('data/hs/hostname').read()
myCore.generateMainPGP(torID)
if os.path.exists('data/pgp/'):
self.assertTrue(True)
def testHMACGen(self):
logger.debug('--------------------------')
logger.info('Running HMAC generation test...')
# Test if hmac key generation is working
import core
myCore = core.Core()
key = myCore.generateHMAC()
if len(key) > 10:
self.assertTrue(True)
else:
def testConfig(self):
logger.debug('-'*26 + '\n')
logger.info('Running simple configuration test...')
import config
config.check()
config.reload()
configdata = str(config.get_config())
config.set('testval', 1337)
if not config.get('testval', None) is 1337:
self.assertTrue(False)
config.set('testval')
if not config.get('testval', None) is None:
self.assertTrue(False)
config.save()
config.reload()
if not str(config.get_config()) == configdata:
self.assertTrue(False)
self.assertTrue(True)
def testBitcoinNode(self):
# temporarily disabled- this takes a lot of time the CI doesn't have
self.assertTrue(True)
#logger.debug('-'*26 + '\n')
#logger.info('Running bitcoin node test...')
#sbitcoin = btc.OnionrBTC()
def testPluginReload(self):
logger.debug('-'*26 + '\n')
logger.info('Running simple plugin reload test...')
import onionrplugins
try:
onionrplugins.reload('test')
self.assertTrue(True)
except:
self.assertTrue(False)
def testPluginStopStart(self):
logger.debug('-'*26 + '\n')
logger.info('Running simple plugin restart test...')
import onionrplugins
try:
onionrplugins.start('test')
onionrplugins.stop('test')
self.assertTrue(True)
except:
self.assertTrue(False)
def testPluginEvent(self):
logger.debug('-'*26 + '\n')
logger.info('Running plugin event test...')
import onionrplugins as plugins, onionrevents as events
plugins.start('test')
if not events.call(plugins.get_plugin('test'), 'test'):
self.assertTrue(False)
events.event('test', data = {'tests': self})
self.assertTrue(True)
def testQueue(self):
logger.debug('--------------------------')
logger.debug('-'*26 + '\n')
logger.info('Running daemon queue test...')
# test if the daemon queue can read/write data
import core
myCore = core.Core()
@ -124,4 +183,32 @@ class OnionrTests(unittest.TestCase):
if command[0] == 'testCommand':
if myCore.daemonQueue() == False:
logger.info('Succesfully added and read command')
def testHashValidation(self):
logger.debug('-'*26 + '\n')
logger.info('Running hash validation test...')
import core
myCore = core.Core()
if not myCore._utils.validateHash("$324dfgfdg") and myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd2") and not myCore._utils.validateHash("f2ca1bb6c7e907d06dafe4687e579fce76b37e4e93b7605022da52e6ccc26fd$"):
self.assertTrue(True)
else:
self.assertTrue(False)
def testAddAdder(self):
logger.debug('-'*26 + '\n')
logger.info('Running address add+remove test')
import core
myCore = core.Core()
if not os.path.exists('data/address.db'):
myCore.createAddressDB()
if myCore.addAddress('facebookcorewwwi.onion') and not myCore.removeAddress('invalid'):
if myCore.removeAddress('facebookcorewwwi.onion'):
self.assertTrue(True)
else:
self.assertTrue(False)
else:
self.assertTrue(False)
unittest.main()

11
onionr/timedHmac.py
View File

@ -16,12 +16,12 @@ import hmac, base64, time, math
class TimedHMAC:
def __init__(self, base64Key, data, hashAlgo):
'''
base64Key = base64 encoded key
data = data to hash
expire = time expiry in epoch
hashAlgo = string in hashlib.algorithms_available
base64Key = base64 encoded key
data = data to hash
expire = time expiry in epoch
hashAlgo = string in hashlib.algorithms_available
Maximum of 10 seconds grace period
Maximum of 10 seconds grace period
'''
self.data = data
self.expire = math.floor(time.time())
@ -30,6 +30,7 @@ class TimedHMAC:
generatedHMAC = hmac.HMAC(base64.b64decode(base64Key).decode(), digestmod=self.hashAlgo)
generatedHMAC.update(data + expire)
self.HMACResult = generatedHMAC.hexdigest()
return
def check(self, data):

24
readme.md
View File

@ -1,18 +1,38 @@
# Onionr
![Onionr logo](./docs/onionr-logo.png)
[![Build Status](https://travis-ci.org/beardog108/onionr.svg?branch=master)](https://travis-ci.org/beardog108/onionr)
[![Open Source Love](https://badges.frapsoft.com/os/v3/open-source.png?v=103)](https://github.com/ellerbrock/open-source-badges/)
P2P microblogging platform and social network, using Tor & I2P.
Anonymous P2P platform, using Tor & I2P.
Major work in progress.
***THIS SOFTWARE IS NOT USABLE OR SAFE YET.***
**Roadmap/features:**
* [X] Fully p2p/decentralized, no trackers or other single points of failure
* [X] High level of anonymity
* [ ] End to end encryption where applicable
* [X] Optional non-encrypted blocks, useful for blog posts or public file sharing
* [ ] Easy API system for integration to websites
# Development
This software is in heavy development. If for some reason you want to get involved, get in touch first.
**Onionr API and functionality is subject to non-backwards compatible change during development**
# Donate
Bitcoin/Bitcoin Cash: 1onion55FXzm6h8KQw3zFw2igpHcV7LPq
## Disclaimer
The Tor Project, I2P developers, and anyone else do not own, create, or endorse this project, and are not otherwise involved.
The badges (besides travis-ci build) are by Maik Ellerbrock is licensed under a Creative Commons Attribution 4.0 International License.
The onion in the Onionr logo is adapted from [this](https://commons.wikimedia.org/wiki/File:Red_Onion_on_White.JPG) image by Colin on Wikimedia under a Creative Commons Attribution-Share Alike 3.0 Unported license

7
requirements.txt
View File

@ -1,11 +1,8 @@
PyNaCl==1.2.1
gnupg==2.3.1
requests==2.12.4
Flask==0.12.2
requests==2.18.4
urllib3==1.22
simple_crypt==4.1.7
urllib3==1.19.1
sha3==0.2.1
pycrypto==2.6.1
pynacl==1.2.1
PySocks==1.6.8
bitpeer.py==0.4.7.5

3
test.sh
View File

@ -1,3 +0,0 @@
#!/bin/sh
cd onionr
./tests.py