Work on normal gossip diffusion

README.md

@@ -8,7 +8,7 @@
     Privacy Respecting Communication Network 📡
 </p>
 <p align="center">
-    Anonymous social platform, mail, file sharing.
+    WIP anonymous social platform, mail, file sharing and marketplace
 </p>
 
 <img src="https://img.shields.io/badge/License-aGPLv3-yellow"> <img src='https://img.shields.io/badge/python%20version%20%F0%9F%90%8D-3.7+-blue'>
@@ -63,20 +63,13 @@ The whitepaper is available [here](docs/whitepaper.md).
 # Main Features
 
 * [X] 🌐 Fully p2p/decentralized, no trackers or other single points of failure
-* [X] 💻 Easy HTTP API for integration to websites
 * [X] 🕵️ Metadata analysis resistance and anonymity
 * [X] 📡 Transport agnosticism (no internet required)
 
 
 # Roadmap
 
-* [ ] 📨 Mail
+See [ROADMAP.md](ROADMAP.md)
-* [ ] 💬 Public message board
-* [ ] 📃 Simple webpage hosting - Will be greatly extended
-* [ ] File sharing (Work in progress)
-* [ ] Store fronts with user reviews
-* [ ] Instant messaging
-
 
 # Documentation
 
@@ -112,7 +105,7 @@ Master may be unstable, you should use the latest release tag. (checkout via git
 * Email: onionr [ at ] voidnet.tech
 * Twitter: [@onionrnet](https://twitter.com/onionrnet)
 * Matrix: #onionr:amorgan.xyz
-* Discord: https://discord.gg/DVF2bEAzrt (Discord is bad for freedom and privacy, this is only provided for convienience)
+* Discord: https://discord.gg/DVF2bEAzrt (Discord is bad for freedom and privacy, this is only provided for convenience)
 
 # Help out
 

ROADMAP.md

@@ -10,16 +10,37 @@
 
     The new system is separated from the underlying networks used and makes it much easier to implement new transport methods. Dandelion++ is also better than the old block mixing we did.
 
+* [  ] Revamped key/encrypted messaging (encrypted blocks)
 * [  ] Restore webUI as a plugin
+* [  ] Restore static site/file sharing plugin
 * [  ] Restore/reimplement mail plugin
 * [  ] Restore/reimplement friends plugin
 * [  ] Refresh test suite
 
 
-## Store Plugin Release (9.1)
+## Misc
+
+* [  ] Spruce up documentation
+* [  ] Restore LAN transport
+
+
+## Web of trust release (10.0)
+
+To facilitate the below market plugin/application, Onionr will need a web of trust system.
+
+* [ ] Web of trust plugin or module
+
+
+## Market Plugin Release (10.1)
+
+The Onionr team believes the Monero community is currently lacking a good p2p market, and as such we hope to build a solution using Onionr as a transport. This may be a separate project and as opposed to a plugin.
+
 
 * A new marketplace plugin will be developed with the following *tenative* features:
 
-* [ ] store front discovery and search
+* [ ] Account management
-* [ ] product listing search
+* [ ] Monero management
-* [ ] user reviews
+* [ ] Store front discovery and search
+* [ ] Product listing search
+* [ ] User reviews
+

src/blockdb/dbpath.py

@@ -0,0 +1,4 @@
+from utils import identifyhome
+
+
+block_db_path = identifyhome.identify_home() + 'blockdata'

src/gossip/client/__init__.py

@@ -70,6 +70,9 @@ def gossip_client():
         get_new_peers,
         1200, initial_sleep=5)
 
+    # Start a new thread to stream blocks from peers
+
+
     dandelion_phase = DandelionPhase(DANDELION_EPOCH_LENGTH)
 
     while True:

src/gossip/client/streamblocks.py

@@ -0,0 +1,39 @@
+"""Onionr - Private P2P Communication.
+
+Download blocks that are being diffused
+
+doesn't apply for blocks in the gossip queue that are awaiting
+descision to fluff or stem
+
+"""
+from random import SystemRandom
+
+if TYPE_CHECKING:
+    from socket import socket
+    from typing import TYPE_CHECKING, List
+
+from ..peerset import gossip_peer_set
+
+"""
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+"""
+
+
+def stream_from_peers():
+    peer_stream_sockets: List[socket] = []
+
+    sys_rand = SystemRandom()
+
+    while True:
+        peers = sys_rand.sample(gossip_peer_set, min(3, len(gossip_peer_set)))

src/gossip/server/__init__.py

@@ -21,7 +21,7 @@ from filepaths import gossip_server_socket_file
 from ..commands import GossipCommands
 from ..peerset import gossip_peer_set
 from .acceptstem import accept_stem_blocks
-from .streamblocks import stream_blocks
+from .diffuseblocks import stream_blocks
 """
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by

src/gossip/server/diffuseblocks.py

@@ -1,6 +1,6 @@
 """Onionr - Private P2P Communication.
 
-Stream blocks we can for inbound edge peers that ask for them
+Diffuse blocks we can for inbound edge peers that ask for them
 
 doesn't apply for blocks in the gossip queue that are awaiting
 descision to fluff or stem
@@ -37,7 +37,7 @@ along with this program.  If not, see <https://www.gnu.org/licenses/>.
 """
 
 
-async def stream_blocks(reader: 'StreamReader', writer: 'StreamWriter'):
+async def diffuse_blocks(reader: 'StreamReader', writer: 'StreamWriter'):
     """stream blocks to a peer created since an offset
     """
     time_offset = await wait_for(reader.read(8), 12)
@@ -54,6 +54,7 @@ async def stream_blocks(reader: 'StreamReader', writer: 'StreamWriter'):
             "Peer's specified time offset skewed too far into the future")
 
     newly_stored_blocks = queue.Queue()
+
     def _add_to_queue(bl):
         newly_stored_blocks.put_nowait(bl)
     block_storage_observers.append(
@@ -70,6 +71,7 @@ async def stream_blocks(reader: 'StreamReader', writer: 'StreamWriter'):
         await writer.drain()
 
     try:
+        # Send initial blocks from offset
         for block in get_blocks_after_timestamp(time_offset):
             if not keep_writing:
                 break
@@ -83,14 +85,15 @@ async def stream_blocks(reader: 'StreamReader', writer: 'StreamWriter'):
             except IncompleteReadError:
                 keep_writing = False
 
+        # Diffuse blocks stored since we started this stream
         while keep_writing:
             await _send_block(newly_stored_blocks.get())
-        try:
+            try:
-            keep_writing = bool(
+                keep_writing = bool(
-                int.from_bytes(await reader.readexactly(1), 'big')
+                    int.from_bytes(await reader.readexactly(1), 'big')
-            )
+                )
-        except IncompleteReadError:
+            except IncompleteReadError:
-            keep_writing = False
+                keep_writing = False
     except Exception:
         logger.warn(traceback.format_exc(), terminal=True)