diff --git a/README.md b/README.md index d02bac2f..197319a0 100644 --- a/README.md +++ b/README.md @@ -11,9 +11,7 @@ Anonymous social platform, mail, file sharing.

- - -current stored block count + @@ -25,16 +23,16 @@
-**The main repository for this software is at https://GitLab.com/beardog/Onionr/** +**The main repository for this software is at https://git.VoidNet.tech/kev/onionr/** -**The [GitHub repository](https://github.com/beardog108/onionr/) is a mirror, do not submit PRs or issues there.** +Mirrors: [Github](https://github.com/beardog108/onionr), [Gitlab](https://gitlab.com/beardog/onionr) Onionr ("Onion Relay") is a decentralized/distributed peer-to-peer communication network, designed to be anonymous and resistant to (meta)data analysis, spam, and corruption. -Onionr stores data in independent packages referred to as 'blocks'. The blocks are distributed to all interested nodes. Blocks and user IDs cannot be easily proven to have been created by a particular user. Even if there is enough evidence to believe that a specific user created a block, nodes still operate behind Tor or I2P and as such cannot be trivially unmasked. Anonymity is achieved by a stateless network, with no given indication of what node a block originates from. Through message mixing and key privacy, it is intended to be nigh impossible to discover the identity of a message creator or recipient. +Onionr stores data in independent packages referred to as 'blocks'. The blocks are distributed to all interested nodes. Blocks and user IDs cannot be easily proven to have been created by a particular user. Even if there is enough evidence to believe that a specific user created a block, nodes still operate behind Tor and as such cannot be trivially unmasked. Anonymity is achieved by a stateless network, with no given indication of what node a block originates from. Through message mixing and key privacy, it is intended to be nigh impossible to discover the identity of a message creator or recipient. -Via long-term traffic analysis, a well funded adversary may discover the most probable node(s) to be creating a set of related blocks, however doing so would only lead them to a node behind Tor or I2P. As the first node that a block appears on is almost always not the creator of the block, there is plausible deniability regarding the true creator of the block. +Via long-term traffic analysis, a well funded adversary may discover the most probable node(s) to be creating a set of related blocks, however doing so would only lead them to a node behind Tor. As the first node that a block appears on is almost always not the creator of the block, there is plausible deniability regarding the true creator of the block. Onionr gives the individual the ability to speak freely, without fear of surveillance and censorship. @@ -139,7 +137,7 @@ Everyone is welcome to contribute. Help is wanted for the following: * UI/UX design * Running stable nodes * Security review/audit -* Automatic I2P setup +* I2P support ## Contribute money: diff --git a/docs/TODO.txt b/docs/TODO.txt index 006f0b02..ed49b915 100644 --- a/docs/TODO.txt +++ b/docs/TODO.txt @@ -12,7 +12,6 @@ * make node "speed" setting such as when ui is open to reduce bandwidth usage * localization support -* add mark read in mail * add BCC support to mail diff --git a/docs/whitepaper.md b/docs/whitepaper.md index d0be643a..b60267e3 100755 --- a/docs/whitepaper.md +++ b/docs/whitepaper.md @@ -4,13 +4,15 @@

Anonymous, Decentralized, Distributed Network

+June 2020 + # Introduction We believe that the ability to communicate freely with others is crucial for maintaining societal and personal liberty. The internet has provided humanity with the ability to spread information globally, but there are many persons and organizations who try to stifle the flow of information, sometimes with success. -Internet censorship comes in many forms, state censorship, corporate consolidation of media, threats of violence, network exploitation (e.g. denial of service attacks) and other threats. +Internet censorship comes in many forms, state censorship, threats of violence, network exploitation (e.g. denial of service attacks) and others. -We hold that in order to protect individual privacy, users must have the ability to communicate anonymously and with decentralization. +We hold that in order to protect individual privacy, users must have the ability to communicate anonymously and with decentralization. We believe that in order to prevent censorship and loss of information, these measures must be in place: @@ -58,12 +60,14 @@ To mitigate maliciously slow or unreliable nodes, Onionr builds a profile on nod ## Block Format -Onionr blocks are very simple. They are structured in two main parts: a metadata section and a data section, with a line feed delimiting where metadata ends and data begins. +Onionr blocks are very simple. They are structured in two main parts: a metadata section and a data section, with a line feed delimiting where metadata ends and data begins. Metadata defines what kind of data is in a block, signature data, encryption settings, and other arbitrary information. Optionally, a random token can be inserted into the metadata for use in Proof of Work. +The proof of work function should be a Verifiable Delay Function (VDF). We have chosen MiMC for it's simplicity. + ### Block Encryption For encryption, Onionr uses ephemeral Curve25519 keys for key exchange and XSalsa20-Poly1305 as a symmetric cipher or optionally using only XSalsa20-Poly1305 with a pre-shared key. @@ -84,9 +88,7 @@ Blocks are stored indefinitely until the allocated space is filled, at which poi ## Block Timestamping -Onionr can provide evidence of when a block was inserted by requesting other users to sign a hash of the current time with the block data hash: sha3_256(time + sha3_256(block data)). - -This can be done either by the creator of the block prior to generation, or by any node after insertion. +Onionr blocks are by default not accepted if their timestamp is set too far in the past, or is in the future. In addition, randomness beacons such as the one operated by [NIST](https://beacon.nist.gov/home), [Chile](https://beacon.clcert.cl/), or the hash of the latest blocks in a cryptocurrency network could be used to affirm that a block was at least not *created* before a given time. @@ -98,7 +100,7 @@ The benefits of such a system are increased privacy, and the ability to anonymou # Threat Model -The goal of Onionr is to provide a method of distributing information in a manner in which the difficulty of discovering the identity of those sending and receiving the information is greatly increased. In this section we detail what information we want to protect and who we're protecting it from. +The goal of Onionr is to provide a method of distributing information in a manner in which the difficulty of discovering the identity of those sending and receiving the information is greatly increased. In this section we detail what information we want to protect and who we're protecting it from. In this threat model, "protected" means available in plaintext only to those which it was intended, and regardless non-malleable @@ -132,7 +134,7 @@ Onionr does not protect the following: ## Assumptions -We assume that Tor onion services (v3) and I2P services cannot be trivially deanonymized, and that the underlying cryptographic primitives we employ cannot be broken in any manner faster than brute force unless a quantum computer is used. +We assume that Tor onion services (v3) and I2P services cannot be trivially deanonymized, and that the underlying cryptographic primitives we employ cannot be broken in any manner faster than brute force unless a quantum computer is used. Once quantum safe algorithms are more mature and have decent high level libraries, they will be deployed. diff --git a/requirements-dev.in b/requirements-dev.in index 52978779..d153a4e5 100644 --- a/requirements-dev.in +++ b/requirements-dev.in @@ -1,3 +1,3 @@ -pdoc3==0.7.5 -pip-tools==4.5.1 -helium==3.0.1 +pdoc3==0.8.3 +pip-tools==5.2.1 +helium==3.0.4 diff --git a/requirements-dev.txt b/requirements-dev.txt index cbbad0e6..d5a0142c 100644 --- a/requirements-dev.txt +++ b/requirements-dev.txt @@ -8,8 +8,9 @@ click==7.0 \ --hash=sha256:2335065e6395b9e67ca716de5f7526736bfa6ceead690adf616d925bdc622b13 \ --hash=sha256:5b94b49521f6456670fdb30cd82a4eca9412788a93fa6dd6df72c94d5a8ff2d7 \ # via pip-tools -helium==3.0.1 \ - --hash=sha256:a7be1ad48702a38e11e9ee9b262459cde3c986d2dcbd79fcf8d6f5b54c5f5d9e +helium==3.0.4 \ + --hash=sha256:035edb4207906fde42f64d47e28dca934327f00fc23c808b45090c2266123998 \ + # via -r requirements-dev.in mako==1.1.1 \ --hash=sha256:2984a6733e1d472796ceef37ad48c26f4a984bb18119bb2dbc37a44d8f6e75a4 \ # via pdoc3 @@ -52,26 +53,27 @@ markupsafe==1.1.1 \ --hash=sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7 \ --hash=sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be \ # via mako +pdoc3==0.8.3 \ + --hash=sha256:19bd1a72e1c82875a6927b244aca826dedb635ea2a7a36ac62cbe063a8ddc30d \ + # via -r requirements-dev.in +pip-tools==5.2.1 \ + --hash=sha256:1690bef5f0f714160c3aedacb03520e2359a78f7f9fa17e574cf8659cf2ef614 \ + --hash=sha256:5b4b6e7b6e66357685c73e856296b4792b2d159ff6074729e250e291834bfd9d \ + # via -r requirements-dev.in selenium==3.141.0 \ --hash=sha256:2d7131d7bc5a5b99a2d9b04aaf2612c411b03b8ca1b1ee8d3de5845a9be2cb3c \ --hash=sha256:deaf32b60ad91a4611b98d8002757f29e6f2c2d5fcaf202e1c9ad06d6772300d \ # via helium -pdoc3==0.7.5 \ - --hash=sha256:ebca75b7fcf23f3b4320abe23339834d3f08c28517718e9d29e555fc38eeb33c \ - # via -r requirements-dev.in -pip-tools==4.5.1 \ - --hash=sha256:693f30e451875796b1b25203247f0b4cf48a4c4a5ab7341f4f33ffd498cdcc98 \ - --hash=sha256:be9c796aa88b2eec5cabf1323ba1cb60a08212b84bfb75b8b4037a8ef8cb8cb6 \ - # via -r requirements-dev.in six==1.14.0 \ --hash=sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a \ --hash=sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c \ # via pip-tools -urllib3==1.25.8 \ - --hash=sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc \ - --hash=sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc \ +urllib3==1.25.9 \ + --hash=sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527 \ + --hash=sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115 \ # via selenium # WARNING: The following packages were not pinned, but pip requires them to be # pinned when the requirements file includes hashes. Consider using the --allow-unsafe flag. +# pip # setuptools diff --git a/requirements.in b/requirements.in index 899df4a8..2284b9c2 100644 --- a/requirements.in +++ b/requirements.in @@ -1,16 +1,17 @@ -urllib3==1.25.8 -requests==2.23.0 -PyNaCl==1.3.0 -gevent==1.4.0 -Flask==1.1.1 +urllib3==1.25.9 +requests==2.24.0 +PyNaCl==1.4.0 +gevent==20.6.2 +Flask==1.1.2 PySocks==1.7.1 stem==1.8.0 deadsimplekv==0.3.1 unpaddedbase32==0.2.0 -streamedrequests==1.0.0 +streamedrequests==1.0.3 toomanyobjs==1.1.0 niceware==0.2.1 -psutil==5.7.0 +psutil==5.7.2 filenuke==0.0.0 -mimcvdf==1.0.0 -watchdog==0.10.2 \ No newline at end of file +mimcvdf==1.1.0 +watchdog==0.10.3 +ujson==3.0.0 diff --git a/requirements.txt b/requirements.txt index 897b0881..65763ac0 100644 --- a/requirements.txt +++ b/requirements.txt @@ -54,55 +54,58 @@ filenuke==0.0.0 \ --hash=sha256:147011c0125121469cae0a8a7f4df399f470e54aa29a08f2d2c099bf0118dcee \ --hash=sha256:c55535dcecfdb27c5f4ce664d46e115950b5429763b5db75c198053646177f8f \ # via -r requirements.in -flask==1.1.1 \ - --hash=sha256:13f9f196f330c7c2c5d7a5cf91af894110ca0215ac051b5844701f2bfd934d52 \ - --hash=sha256:45eb5a6fd193d6cf7e0cf5d8a5b31f83d5faae0293695626f539a823e93b13f6 \ +flask==1.1.2 \ + --hash=sha256:4efa1ae2d7c9865af48986de8aeb8504bf32c7f3d6fdc9353d34b21f4b127060 \ + --hash=sha256:8a4fdd8936eba2512e9c85df320a37e694c93945b33ef33c89946a340a238557 \ # via -r requirements.in -gevent==1.4.0 \ - --hash=sha256:0774babec518a24d9a7231d4e689931f31b332c4517a771e532002614e270a64 \ - --hash=sha256:0e1e5b73a445fe82d40907322e1e0eec6a6745ca3cea19291c6f9f50117bb7ea \ - --hash=sha256:0ff2b70e8e338cf13bedf146b8c29d475e2a544b5d1fe14045aee827c073842c \ - --hash=sha256:107f4232db2172f7e8429ed7779c10f2ed16616d75ffbe77e0e0c3fcdeb51a51 \ - --hash=sha256:14b4d06d19d39a440e72253f77067d27209c67e7611e352f79fe69e0f618f76e \ - --hash=sha256:1b7d3a285978b27b469c0ff5fb5a72bcd69f4306dbbf22d7997d83209a8ba917 \ - --hash=sha256:1eb7fa3b9bd9174dfe9c3b59b7a09b768ecd496debfc4976a9530a3e15c990d1 \ - --hash=sha256:2711e69788ddb34c059a30186e05c55a6b611cb9e34ac343e69cf3264d42fe1c \ - --hash=sha256:28a0c5417b464562ab9842dd1fb0cc1524e60494641d973206ec24d6ec5f6909 \ - --hash=sha256:3249011d13d0c63bea72d91cec23a9cf18c25f91d1f115121e5c9113d753fa12 \ - --hash=sha256:44089ed06a962a3a70e96353c981d628b2d4a2f2a75ea5d90f916a62d22af2e8 \ - --hash=sha256:4bfa291e3c931ff3c99a349d8857605dca029de61d74c6bb82bd46373959c942 \ - --hash=sha256:50024a1ee2cf04645535c5ebaeaa0a60c5ef32e262da981f4be0546b26791950 \ - --hash=sha256:53b72385857e04e7faca13c613c07cab411480822ac658d97fd8a4ddbaf715c8 \ - --hash=sha256:74b7528f901f39c39cdbb50cdf08f1a2351725d9aebaef212a29abfbb06895ee \ - --hash=sha256:7d0809e2991c9784eceeadef01c27ee6a33ca09ebba6154317a257353e3af922 \ - --hash=sha256:896b2b80931d6b13b5d9feba3d4eebc67d5e6ec54f0cf3339d08487d55d93b0e \ - --hash=sha256:8d9ec51cc06580f8c21b41fd3f2b3465197ba5b23c00eb7d422b7ae0380510b0 \ - --hash=sha256:9f7a1e96fec45f70ad364e46de32ccacab4d80de238bd3c2edd036867ccd48ad \ - --hash=sha256:ab4dc33ef0e26dc627559786a4fba0c2227f125db85d970abbf85b77506b3f51 \ - --hash=sha256:d1e6d1f156e999edab069d79d890859806b555ce4e4da5b6418616322f0a3df1 \ - --hash=sha256:d752bcf1b98174780e2317ada12013d612f05116456133a6acf3e17d43b71f05 \ - --hash=sha256:e5bcc4270671936349249d26140c267397b7b4b1381f5ec8b13c53c5b53ab6e1 \ +gevent==20.6.2 \ + --hash=sha256:0b16dd85eddaf6acdad373ce90ed4da09ef466cbc5e0ee5932d13f099929e844 \ + --hash=sha256:0f3fbb1703b10609856e5dffb0e358bf5edf57e52dc7cd7226e3f8674fdc0a0f \ + --hash=sha256:13c74d6784ef5ada2666abf2bb310d27a1d14291f7cac46148f336b19f714d40 \ + --hash=sha256:1ea0d34cb78cdf37870be3bfb9330ebda89197bed9e048c14f4a90dec19a33e0 \ + --hash=sha256:354f932c284fa45826b32f42927d892096cce05671b50b3ff59528230217ad47 \ + --hash=sha256:3cb2f6978615d52e4e4e667b035c11a7272bb68b14d119faf1b138164b2f354f \ + --hash=sha256:67776cb33b638a3c61a0351d9d1e8f33a46b47de619e249de1159892f9ff035c \ + --hash=sha256:68764aca061bbbbade43727e797f9c28042f6d90cca5fb6514ef726d43ab00ca \ + --hash=sha256:6c864b5604166ac8351e3128a1135b883b9e978fd24afbd75a249dcb42bc8ab5 \ + --hash=sha256:73eb4cf3114fbb5dd801bd0b93941adfa2fa6d99e91976c20a121ea14b8b39b9 \ + --hash=sha256:76ef4c6e3332e6f7278142d791b28695adfce39735900fccef2a0f1d894f6b36 \ + --hash=sha256:78bd94f6f2ac366155169df3507068f6381f2ad77625633189ce183f86a57597 \ + --hash=sha256:7d8408854ce892f987305a0e9bf5c051f4ea29453665454396d6afb620c719b6 \ + --hash=sha256:9527087984f1659be899b3300d5d61c7c5b01d8beae106aff5160316da8bc56f \ + --hash=sha256:a18d8dd9bfa994a22f30adfa0563d80f0809140045c34f85535f422813d25855 \ + --hash=sha256:a23c2abf08e851c988723f6a2996d495f513a2c0dc70f9956af03af8debdb5d1 \ + --hash=sha256:a47556cac07e31b3cef8fd701599b3b1365961fe3736471f41807ffa27c5c848 \ + --hash=sha256:b03890bbddbae5667f5baad517417056496ff5e92c3c7945b27cc08f55a9fcb2 \ + --hash=sha256:b17915b65b49a425115ddc3087484c81b1e47ce38c931d18bb14e453753e4d06 \ + --hash=sha256:bef18b8bd3b728240b9bbd699737216b793d6c97b482431f69dcbe328ad73692 \ + --hash=sha256:c0f4340e40e0f9dfe93a52a12ddf5b1eeda9bbc89b99bf3b9b23acab0dfae0a4 \ + --hash=sha256:d0a67a20ce325f6a2068e0bd9fbf83db8a5f5ced972ed8ac5c20079a7d98c7d1 \ + --hash=sha256:d3baff87d935a5eeffb0e4f7cd5ffe258d2430cd62aeee2e5396f85da07df435 \ + --hash=sha256:e5ca5ee80a9d9e697c9fc22b4bbce9ad06870f83fc8e7774e5504892ef702476 \ + --hash=sha256:ea2e4584950186b71d648bde6af40dae4d4c6f43db25a732ec056b27a7a83afe \ + --hash=sha256:ebb8a545112110e3a6edf905ae1556b0538fc148c743aa7d8cfaebbbc23de31d \ + --hash=sha256:f2a02d9004ccb18edd9eaf6f25da9a7763de41a69754d5e4d872a8cbf8bd0b72 \ + --hash=sha256:f41cc8e853ac2252bc58f6feabd74b8aae613e2d19097c5373463122f4dc08f0 \ # via -r requirements.in -greenlet==0.4.15 \ - --hash=sha256:000546ad01e6389e98626c1367be58efa613fa82a1be98b0c6fc24b563acc6d0 \ - --hash=sha256:0d48200bc50cbf498716712129eef819b1729339e34c3ae71656964dac907c28 \ - --hash=sha256:23d12eacffa9d0f290c0fe0c4e81ba6d5f3a5b7ac3c30a5eaf0126bf4deda5c8 \ - --hash=sha256:37c9ba82bd82eb6a23c2e5acc03055c0e45697253b2393c9a50cef76a3985304 \ - --hash=sha256:51503524dd6f152ab4ad1fbd168fc6c30b5795e8c70be4410a64940b3abb55c0 \ - --hash=sha256:8041e2de00e745c0e05a502d6e6db310db7faa7c979b3a5877123548a4c0b214 \ - --hash=sha256:81fcd96a275209ef117e9ec91f75c731fa18dcfd9ffaa1c0adbdaa3616a86043 \ - --hash=sha256:853da4f9563d982e4121fed8c92eea1a4594a2299037b3034c3c898cb8e933d6 \ - --hash=sha256:8b4572c334593d449113f9dc8d19b93b7b271bdbe90ba7509eb178923327b625 \ - --hash=sha256:9416443e219356e3c31f1f918a91badf2e37acf297e2fa13d24d1cc2380f8fbc \ - --hash=sha256:9854f612e1b59ec66804931df5add3b2d5ef0067748ea29dc60f0efdcda9a638 \ - --hash=sha256:99a26afdb82ea83a265137a398f570402aa1f2b5dfb4ac3300c026931817b163 \ - --hash=sha256:a19bf883b3384957e4a4a13e6bd1ae3d85ae87f4beb5957e35b0be287f12f4e4 \ - --hash=sha256:a9f145660588187ff835c55a7d2ddf6abfc570c2651c276d3d4be8a2766db490 \ - --hash=sha256:ac57fcdcfb0b73bb3203b58a14501abb7e5ff9ea5e2edfa06bb03035f0cff248 \ - --hash=sha256:bcb530089ff24f6458a81ac3fa699e8c00194208a724b644ecc68422e1111939 \ - --hash=sha256:beeabe25c3b704f7d56b573f7d2ff88fc99f0138e43480cecdfcaa3b87fe4f87 \ - --hash=sha256:d634a7ea1fc3380ff96f9e44d8d22f38418c1c381d5fac680b272d7d90883720 \ - --hash=sha256:d97b0661e1aead761f0ded3b769044bb00ed5d33e1ec865e891a8b128bf7c656 \ +greenlet==0.4.16 \ + --hash=sha256:1000038ba0ea9032948e2156a9c15f5686f36945e8f9906e6b8db49f358e7b52 \ + --hash=sha256:133ba06bad4e5f2f8bf6a0ac434e0fd686df749a86b3478903b92ec3a9c0c90b \ + --hash=sha256:1429dc183b36ec972055e13250d96e174491559433eb3061691b446899b87384 \ + --hash=sha256:1b805231bfb7b2900a16638c3c8b45c694334c811f84463e52451e00c9412691 \ + --hash=sha256:3a35e33902b2e6079949feed7a2dafa5ac6f019da97bd255842bb22de3c11bf5 \ + --hash=sha256:5ea034d040e6ab1d2ae04ab05a3f37dbd719c4dee3804b13903d4cc794b1336e \ + --hash=sha256:682328aa576ec393c1872615bcb877cf32d800d4a2f150e1a5dc7e56644010b1 \ + --hash=sha256:6e06eac722676797e8fce4adb8ad3dc57a1bb3adfb0dd3fdf8306c055a38456c \ + --hash=sha256:7eed31f4efc8356e200568ba05ad645525f1fbd8674f1e5be61a493e715e3873 \ + --hash=sha256:80cb0380838bf4e48da6adedb0c7cd060c187bb4a75f67a5aa9ec33689b84872 \ + --hash=sha256:b0b2a984bbfc543d144d88caad6cc7ff4a71be77102014bd617bd88cfb038727 \ + --hash=sha256:c196a5394c56352e21cb7224739c6dd0075b69dd56f758505951d1d8d68cf8a9 \ + --hash=sha256:d83c1d38658b0f81c282b41238092ed89d8f93c6e342224ab73fb39e16848721 \ + --hash=sha256:df7de669cbf21de4b04a3ffc9920bc8426cab4c61365fa84d79bf97401a8bef7 \ + --hash=sha256:e5db19d4a7d41bbeb3dd89b49fc1bc7e6e515b51bbf32589c618655a0ebe0bf0 \ + --hash=sha256:e695ac8c3efe124d998230b219eb51afb6ef10524a50b3c45109c4b77a8a3a92 \ + --hash=sha256:eac2a3f659d5f41d6bbfb6a97733bc7800ea5e906dc873732e00cebb98cec9e4 \ # via gevent idna==2.7 \ --hash=sha256:156a6814fb5ac1fc6850fb002e0852d56c0c8d2531923a51032d1b70760e186e \ @@ -146,9 +149,9 @@ markupsafe==1.1.1 \ --hash=sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f \ --hash=sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7 \ # via jinja2 -mimcvdf==1.0.0 \ - --hash=sha256:4675261d3a49c5665063fdaf256765d160617ca4301601aa1450e98bb96afbbc \ - --hash=sha256:c0d4a79a52078c2ec13c6a5eaa086681a947669708a9b9a2e3b6cd6518c59371 \ +mimcvdf==1.1.0 \ + --hash=sha256:97a4ccdebb58352c64c268d2e57ef8817c9fe4ac3dcc922410bfcc72033f344a \ + --hash=sha256:ae47c79bfd6b7b76077c8ce3301a48a7c10a609d8a882e7bd785e2ef851ecd28 \ # via -r requirements.in niceware==0.2.1 \ --hash=sha256:0f8b192f2a1e800e068474f6e208be9c7e2857664b33a96f4045340de4e5c69c \ @@ -157,51 +160,38 @@ niceware==0.2.1 \ pathtools==0.1.2 \ --hash=sha256:7c35c5421a39bb82e58018febd90e3b6e5db34c5443aaaf742b3f33d4655f1c0 \ # via watchdog -psutil==5.7.0 \ - --hash=sha256:1413f4158eb50e110777c4f15d7c759521703bd6beb58926f1d562da40180058 \ - --hash=sha256:298af2f14b635c3c7118fd9183843f4e73e681bb6f01e12284d4d70d48a60953 \ - --hash=sha256:60b86f327c198561f101a92be1995f9ae0399736b6eced8f24af41ec64fb88d4 \ - --hash=sha256:685ec16ca14d079455892f25bd124df26ff9137664af445563c1bd36629b5e0e \ - --hash=sha256:73f35ab66c6c7a9ce82ba44b1e9b1050be2a80cd4dcc3352cc108656b115c74f \ - --hash=sha256:75e22717d4dbc7ca529ec5063000b2b294fc9a367f9c9ede1f65846c7955fd38 \ - --hash=sha256:a02f4ac50d4a23253b68233b07e7cdb567bd025b982d5cf0ee78296990c22d9e \ - --hash=sha256:d008ddc00c6906ec80040d26dc2d3e3962109e40ad07fd8a12d0284ce5e0e4f8 \ - --hash=sha256:d84029b190c8a66a946e28b4d3934d2ca1528ec94764b180f7d6ea57b0e75e26 \ - --hash=sha256:e2d0c5b07c6fe5a87fa27b7855017edb0d52ee73b71e6ee368fae268605cc3f5 \ - --hash=sha256:f344ca230dd8e8d5eee16827596f1c22ec0876127c28e800d7ae20ed44c4b310 \ +psutil==5.7.2 \ + --hash=sha256:0ee3c36428f160d2d8fce3c583a0353e848abb7de9732c50cf3356dd49ad63f8 \ + --hash=sha256:10512b46c95b02842c225f58fa00385c08fa00c68bac7da2d9a58ebe2c517498 \ + --hash=sha256:4080869ed93cce662905b029a1770fe89c98787e543fa7347f075ade761b19d6 \ + --hash=sha256:5e9d0f26d4194479a13d5f4b3798260c20cecf9ac9a461e718eb59ea520a360c \ + --hash=sha256:66c18ca7680a31bf16ee22b1d21b6397869dda8059dbdb57d9f27efa6615f195 \ + --hash=sha256:68d36986ded5dac7c2dcd42f2682af1db80d4bce3faa126a6145c1637e1b559f \ + --hash=sha256:90990af1c3c67195c44c9a889184f84f5b2320dce3ee3acbd054e3ba0b4a7beb \ + --hash=sha256:a5b120bb3c0c71dfe27551f9da2f3209a8257a178ed6c628a819037a8df487f1 \ + --hash=sha256:d8a82162f23c53b8525cf5f14a355f5d1eea86fa8edde27287dd3a98399e4fdf \ + --hash=sha256:f2018461733b23f308c298653c8903d32aaad7873d25e1d228765e91ae42c3f2 \ + --hash=sha256:ff1977ba1a5f71f89166d5145c3da1cea89a0fdb044075a12c720ee9123ec818 \ # via -r requirements.in pycparser==2.19 \ --hash=sha256:a988718abfad80b6b157acce7bf130a30876d27603738ac39f140993246b25b3 \ # via cffi -pynacl==1.3.0 \ - --hash=sha256:05c26f93964373fc0abe332676cb6735f0ecad27711035b9472751faa8521255 \ - --hash=sha256:0c6100edd16fefd1557da078c7a31e7b7d7a52ce39fdca2bec29d4f7b6e7600c \ - --hash=sha256:0d0a8171a68edf51add1e73d2159c4bc19fc0718e79dec51166e940856c2f28e \ - --hash=sha256:1c780712b206317a746ace34c209b8c29dbfd841dfbc02aa27f2084dd3db77ae \ - --hash=sha256:2424c8b9f41aa65bbdbd7a64e73a7450ebb4aa9ddedc6a081e7afcc4c97f7621 \ - --hash=sha256:2d23c04e8d709444220557ae48ed01f3f1086439f12dbf11976e849a4926db56 \ - --hash=sha256:30f36a9c70450c7878053fa1344aca0145fd47d845270b43a7ee9192a051bf39 \ - --hash=sha256:37aa336a317209f1bb099ad177fef0da45be36a2aa664507c5d72015f956c310 \ - --hash=sha256:4943decfc5b905748f0756fdd99d4f9498d7064815c4cf3643820c9028b711d1 \ - --hash=sha256:57ef38a65056e7800859e5ba9e6091053cd06e1038983016effaffe0efcd594a \ - --hash=sha256:5bd61e9b44c543016ce1f6aef48606280e45f892a928ca7068fba30021e9b786 \ - --hash=sha256:6482d3017a0c0327a49dddc8bd1074cc730d45db2ccb09c3bac1f8f32d1eb61b \ - --hash=sha256:7d3ce02c0784b7cbcc771a2da6ea51f87e8716004512493a2b69016326301c3b \ - --hash=sha256:a14e499c0f5955dcc3991f785f3f8e2130ed504fa3a7f44009ff458ad6bdd17f \ - --hash=sha256:a39f54ccbcd2757d1d63b0ec00a00980c0b382c62865b61a505163943624ab20 \ - --hash=sha256:aabb0c5232910a20eec8563503c153a8e78bbf5459490c49ab31f6adf3f3a415 \ - --hash=sha256:bd4ecb473a96ad0f90c20acba4f0bf0df91a4e03a1f4dd6a4bdc9ca75aa3a715 \ - --hash=sha256:e2da3c13307eac601f3de04887624939aca8ee3c9488a0bb0eca4fb9401fc6b1 \ - --hash=sha256:f67814c38162f4deb31f68d590771a29d5ae3b1bd64b75cf232308e5c74777e0 \ +pynacl==1.4.0 \ + --hash=sha256:30f9b96db44e09b3304f9ea95079b1b7316b2b4f3744fe3aaecccd95d547063d \ + --hash=sha256:54e9a2c849c742006516ad56a88f5c74bf2ce92c9f67435187c3c5953b346505 \ + --hash=sha256:7757ae33dae81c300487591c68790dfb5145c7d03324000433d9a2c141f82af7 \ + --hash=sha256:95be540c5a3359019d999c91d6ff0b59c6c30f85a70b8860a885fdf1a3f92840 \ + --hash=sha256:c878f2a3f9ccbce71167da0847797198138af35a7ae3f8673e80d7a6c5336d94 \ + --hash=sha256:d452a6746f0a7e11121e64625109bc4468fc3100452817001dbe018bb8b08514 \ # via -r requirements.in pysocks==1.7.1 \ --hash=sha256:08e69f092cc6dbe92a0fdd16eeb9b9ffbc13cadfe5ca4c7bd92ffb078b293299 \ --hash=sha256:2725bd0a9925919b9b51739eea5f9e2bae91e83288108a9ad338b2e3a4435ee5 \ --hash=sha256:3f8804571ebe159c380ac6de37643bb4685970655d3bba243530d6558b799aa0 \ # via -r requirements.in -requests==2.23.0 \ - --hash=sha256:43999036bfa82904b6af1d99e4882b560e5e2c68e5c4b0aa03b655f3d7d73fee \ - --hash=sha256:b3f43d496c6daba4493e7c431722aeb7dbc6288f52a6e04e7b6023b0247817e6 \ +requests==2.24.0 \ + --hash=sha256:b3559a131db72c33ee969480840fff4bb6dd111de7dd27c8ee1f820f4f00231b \ + --hash=sha256:fe75cc94a9443b9246fc7049224f75604b113c36acb93f87b80ed42c44cbb898 \ # via -r requirements.in, streamedrequests six==1.12.0 \ --hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \ @@ -210,24 +200,90 @@ six==1.12.0 \ stem==1.8.0 \ --hash=sha256:a0b48ea6224e95f22aa34c0bc3415f0eb4667ddeae3dfb5e32a6920c185568c2 \ # via -r requirements.in -streamedrequests==1.0.0 \ - --hash=sha256:1d9d07394804a6e1fd66bde74a804e71cab98e6920053865574a459f1cf7d3b7 \ +streamedrequests==1.0.3 \ + --hash=sha256:4388ffc0ee94dda719dafc4324b8ddd108cb2231ec59871de79e2592bf4eef0a \ + --hash=sha256:ee68417a1522e75c35b1b2d5f3b6f7e76a3a1a6c0ef5e0c573d08307910079d8 \ # via -r requirements.in toomanyobjs==1.1.0 \ --hash=sha256:99e27468f9dad19127be9e2fb086b42acd69aed9ad7e63cef74d6e4389be0534 \ # via -r requirements.in +ujson==3.0.0 \ + --hash=sha256:019a17e7162f26e264f1645bb41630f7103d178c092ea4bb8f3b16126c3ea210 \ + --hash=sha256:0379ffc7484b862a292e924c15ad5f1c5306d4271e2efd162144812afb08ff97 \ + --hash=sha256:0959a5b569e192459b492b007e3fd63d8f4b4bcb4f69dcddca850a9b9dfe3e7a \ + --hash=sha256:0e2352b60c4ac4fc75b723435faf36ef5e7f3bfb988adb4d589b5e0e6e1d90aa \ + --hash=sha256:0f33359908df32033195bfdd59ba2bfb90a23cb280ef9a0ba11e5013a53d7fd9 \ + --hash=sha256:154f778f0b028390067aaedce8399730d4f528a16a1c214fe4eeb9c4e4f51810 \ + --hash=sha256:3bd791d17a175c1c6566aeaec1755b58e3f021fe9bb62f10f02b656b299199f5 \ + --hash=sha256:634c206f4fb3be7e4523768c636d2dd41cb9c7130e2d219ef8305b8fb6f4838e \ + --hash=sha256:670018d4ab4b0755a7234a9f4791723abcd0506c0eed33b2ed50579c4aff31f2 \ + --hash=sha256:9c68557da3e3ad57e0105aceba0cce5f8f7cd07d207c3860e59c0b3044532830 \ + --hash=sha256:a32f2def62b10e8a19084d17d40363c4da1ac5f52d300a9e99d7efb49fe5f34a \ + --hash=sha256:bea2958c7b5bf4f191f0def751b6f7c8b208edb5f7277e21776329f2ca042385 \ + --hash=sha256:c04d253fec814657fd9f150ef2333dbd0bc6f46208355aa753a29e0696b7fa7e \ + --hash=sha256:c841a6450d64c24c64cbcca429bab22cdb6daef5eaddfdfebe798a5e9e5aff4c \ + --hash=sha256:e0199849d61cc6418f94d52a314c6a27524d65e82174d2a043fb718f73d1520d \ + --hash=sha256:f40bb0d0cb534aad3e24884cf864bda7a71eb5984bd1da61d1711bbfb3be2c38 \ + --hash=sha256:f854702a9aff3a445f4a0b715d240f2a3d84014d8ae8aad05a982c7ffab12525 \ + # via -r requirements.in unpaddedbase32==0.2.0 \ --hash=sha256:4aacee75f8fd6c8cf129842ecba45ca59c11bfb13dae19d86f32b48fa3715403 \ --hash=sha256:b7b780c31d27d55e66abf6c221216a35690ee8892c2daacff7f2528e229bd9c3 \ # via -r requirements.in -urllib3==1.25.8 \ - --hash=sha256:2f3db8b19923a873b3e5256dc9c2dedfa883e33d87c690d9c7913e1f40673cdc \ - --hash=sha256:87716c2d2a7121198ebcb7ce7cccf6ce5e9ba539041cfbaeecfb641dc0bf6acc \ +urllib3==1.25.9 \ + --hash=sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527 \ + --hash=sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115 \ # via -r requirements.in, requests -watchdog==0.10.2 \ - --hash=sha256:c560efb643faed5ef28784b2245cf8874f939569717a4a12826a173ac644456b \ +watchdog==0.10.3 \ + --hash=sha256:4214e1379d128b0588021880ccaf40317ee156d4603ac388b9adcf29165e0c04 \ # via -r requirements.in werkzeug==0.15.5 \ --hash=sha256:87ae4e5b5366da2347eb3116c0e6c681a0e939a33b2805e2c0cbd282664932c4 \ --hash=sha256:a13b74dd3c45f758d4ebdb224be8f1ab8ef58b3c0ffc1783a8c7d9f4f50227e6 \ # via flask +zope.event==4.4 \ + --hash=sha256:69c27debad9bdacd9ce9b735dad382142281ac770c4a432b533d6d65c4614bcf \ + --hash=sha256:d8e97d165fd5a0997b45f5303ae11ea3338becfe68c401dd88ffd2113fe5cae7 \ + # via gevent +zope.interface==5.1.0 \ + --hash=sha256:0103cba5ed09f27d2e3de7e48bb320338592e2fabc5ce1432cf33808eb2dfd8b \ + --hash=sha256:14415d6979356629f1c386c8c4249b4d0082f2ea7f75871ebad2e29584bd16c5 \ + --hash=sha256:1ae4693ccee94c6e0c88a4568fb3b34af8871c60f5ba30cf9f94977ed0e53ddd \ + --hash=sha256:1b87ed2dc05cb835138f6a6e3595593fea3564d712cb2eb2de963a41fd35758c \ + --hash=sha256:269b27f60bcf45438e8683269f8ecd1235fa13e5411de93dae3b9ee4fe7f7bc7 \ + --hash=sha256:27d287e61639d692563d9dab76bafe071fbeb26818dd6a32a0022f3f7ca884b5 \ + --hash=sha256:39106649c3082972106f930766ae23d1464a73b7d30b3698c986f74bf1256a34 \ + --hash=sha256:40e4c42bd27ed3c11b2c983fecfb03356fae1209de10686d03c02c8696a1d90e \ + --hash=sha256:461d4339b3b8f3335d7e2c90ce335eb275488c587b61aca4b305196dde2ff086 \ + --hash=sha256:4f98f70328bc788c86a6a1a8a14b0ea979f81ae6015dd6c72978f1feff70ecda \ + --hash=sha256:558a20a0845d1a5dc6ff87cd0f63d7dac982d7c3be05d2ffb6322a87c17fa286 \ + --hash=sha256:562dccd37acec149458c1791da459f130c6cf8902c94c93b8d47c6337b9fb826 \ + --hash=sha256:5e86c66a6dea8ab6152e83b0facc856dc4d435fe0f872f01d66ce0a2131b7f1d \ + --hash=sha256:60a207efcd8c11d6bbeb7862e33418fba4e4ad79846d88d160d7231fcb42a5ee \ + --hash=sha256:645a7092b77fdbc3f68d3cc98f9d3e71510e419f54019d6e282328c0dd140dcd \ + --hash=sha256:6874367586c020705a44eecdad5d6b587c64b892e34305bb6ed87c9bbe22a5e9 \ + --hash=sha256:74bf0a4f9091131de09286f9a605db449840e313753949fe07c8d0fe7659ad1e \ + --hash=sha256:7b726194f938791a6691c7592c8b9e805fc6d1b9632a833b9c0640828cd49cbc \ + --hash=sha256:8149ded7f90154fdc1a40e0c8975df58041a6f693b8f7edcd9348484e9dc17fe \ + --hash=sha256:8cccf7057c7d19064a9e27660f5aec4e5c4001ffcf653a47531bde19b5aa2a8a \ + --hash=sha256:911714b08b63d155f9c948da2b5534b223a1a4fc50bb67139ab68b277c938578 \ + --hash=sha256:a5f8f85986197d1dd6444763c4a15c991bfed86d835a1f6f7d476f7198d5f56a \ + --hash=sha256:a744132d0abaa854d1aad50ba9bc64e79c6f835b3e92521db4235a1991176813 \ + --hash=sha256:af2c14efc0bb0e91af63d00080ccc067866fb8cbbaca2b0438ab4105f5e0f08d \ + --hash=sha256:b054eb0a8aa712c8e9030065a59b5e6a5cf0746ecdb5f087cca5ec7685690c19 \ + --hash=sha256:b0becb75418f8a130e9d465e718316cd17c7a8acce6fe8fe07adc72762bee425 \ + --hash=sha256:b1d2ed1cbda2ae107283befd9284e650d840f8f7568cb9060b5466d25dc48975 \ + --hash=sha256:ba4261c8ad00b49d48bbb3b5af388bb7576edfc0ca50a49c11dcb77caa1d897e \ + --hash=sha256:d1fe9d7d09bb07228650903d6a9dc48ea649e3b8c69b1d263419cc722b3938e8 \ + --hash=sha256:d7804f6a71fc2dda888ef2de266727ec2f3915373d5a785ed4ddc603bbc91e08 \ + --hash=sha256:da2844fba024dd58eaa712561da47dcd1e7ad544a257482392472eae1c86d5e5 \ + --hash=sha256:dcefc97d1daf8d55199420e9162ab584ed0893a109f45e438b9794ced44c9fd0 \ + --hash=sha256:dd98c436a1fc56f48c70882cc243df89ad036210d871c7427dc164b31500dc11 \ + --hash=sha256:e74671e43ed4569fbd7989e5eecc7d06dc134b571872ab1d5a88f4a123814e9f \ + --hash=sha256:eb9b92f456ff3ec746cd4935b73c1117538d6124b8617bc0fe6fda0b3816e345 \ + --hash=sha256:ebb4e637a1fb861c34e48a00d03cffa9234f42bef923aec44e5625ffb9a8e8f9 \ + --hash=sha256:ef739fe89e7f43fb6494a43b1878a36273e5924869ba1d866f752c5812ae8d58 \ + --hash=sha256:f40db0e02a8157d2b90857c24d89b6310f9b6c3642369852cdc3b5ac49b92afc \ + --hash=sha256:f68bf937f113b88c866d090fea0bc52a098695173fc613b055a17ff0cf9683b6 \ + --hash=sha256:fb55c182a3f7b84c1a2d6de5fa7b1a05d4660d866b91dbf8d74549c57a1499e8 \ + # via gevent diff --git a/scripts/disable-dev-config.py b/scripts/disable-dev-config.py index 685b21ab..4b7e34cc 100755 --- a/scripts/disable-dev-config.py +++ b/scripts/disable-dev-config.py @@ -26,6 +26,7 @@ conf['transports']['tor'] = True conf['transports']['sneakernet'] = True conf['statistics']['i_dont_want_privacy'] = False conf['statistics']['server'] = '' +conf['ui']['animated_background'] = True json.dump(conf, open('static-data/default_config.json', 'w'), sort_keys=True, indent=4) diff --git a/scripts/enable-dev-config.py b/scripts/enable-dev-config.py index d547843e..152f1b43 100755 --- a/scripts/enable-dev-config.py +++ b/scripts/enable-dev-config.py @@ -27,6 +27,7 @@ conf['onboarding']['done'] = True conf['general']['minimum_block_pow'] = block_pow conf['general']['minimum_send_pow'] = block_pow conf['log']['file']['remove_on_exit'] = False +conf['ui']['animated_background'] = False if input('Stat reporting? y/n') == 'y': conf['statistics']['i_dont_want_privacy'] = True conf['statistics']['server'] = input('Statistics server') diff --git a/src/__init__.py b/src/__init__.py index 8fd207cf..c53609c2 100755 --- a/src/__init__.py +++ b/src/__init__.py @@ -5,6 +5,13 @@ This file initializes Onionr when ran to be a daemon or with commands Run with 'help' for usage. """ +import sys +try: + import sqlite3 +except ModuleNotFoundError: + sys.stderr.write( + 'Error, Onionr requires Sqlite3-enabled Python.\n') + sys.exit(1) """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -28,12 +35,12 @@ ran_as_script = False if __name__ == "__main__": ran_as_script = True # Import standard libraries -import sys # noqa try: from etc import dependencycheck # noqa except ModuleNotFoundError as e: - print('Onionr needs ' + str(e) + ' installed') + print('Missing requirement: ' + str(e) + ' installed') + sys.exit(1) # Import 3rd party libraries diff --git a/src/bigbrother/ministry/ofdisk.py b/src/bigbrother/ministry/ofdisk.py index 957e4a69..136e8f09 100644 --- a/src/bigbrother/ministry/ofdisk.py +++ b/src/bigbrother/ministry/ofdisk.py @@ -9,7 +9,6 @@ def detect_disk_access(info): whitelist = [identify_home(), 'onionr/src/', '/site-packages/', '/usr/lib64/'] - for item in whitelist: if item in info[0]: return diff --git a/src/communicator/__init__.py b/src/communicator/__init__.py index d31a40fc..c316275c 100755 --- a/src/communicator/__init__.py +++ b/src/communicator/__init__.py @@ -26,6 +26,7 @@ from communicatorutils import housekeeping from communicatorutils import netcheck from onionrutils import localcommand from onionrutils import epoch +from onionrcommands.openwebinterface import get_url from etc import humanreadabletime import onionrservices import filepaths @@ -241,6 +242,8 @@ class OnionrCommunicatorDaemon: logger.info( 'First run detected. Run openhome to get setup.', terminal=True) + get_url() + while not config.get('onboarding.done', True) and \ not self.shutdown: diff --git a/src/communicator/onlinepeers/onlinepeers.py b/src/communicator/onlinepeers/onlinepeers.py index a37ba689..d422c871 100644 --- a/src/communicator/onlinepeers/onlinepeers.py +++ b/src/communicator/onlinepeers/onlinepeers.py @@ -55,6 +55,10 @@ def get_online_peers(comm_inst: 'OnionrCommunicatorDaemon'): if len(comm_inst.onlinePeers) == 0: logger.debug('Couldn\'t connect to any peers.' + f' Last node seen {last_seen} ago.') + try: + get_online_peers(comm_inst) + except RecursionError: + pass else: comm_inst.lastNodeSeen = time.time() comm_inst.decrementThreadCount('get_online_peers') diff --git a/src/communicatorutils/announcenode.py b/src/communicatorutils/announcenode.py index a068b88c..f8e45a81 100755 --- a/src/communicatorutils/announcenode.py +++ b/src/communicatorutils/announcenode.py @@ -1,14 +1,11 @@ """ - Onionr - Private P2P Communication. +Onionr - Private P2P Communication. Use a communicator instance to announce our transport address to connected nodes """ -import base64 -import onionrproofs import logger -from etc import onionrvalues -from onionrutils import basicrequests, bytesconverter +from onionrutils import basicrequests from utils import gettransports from netcontroller import NetController from communicator import onlinepeers @@ -33,7 +30,6 @@ import onionrexceptions def announce_node(daemon): """Announce our node to our peers.""" ret_data = False - announce_fail = False # Do not let announceCache get too large if len(daemon.announceCache) >= 10000: diff --git a/src/communicatorutils/connectnewpeers.py b/src/communicatorutils/connectnewpeers.py index c9a60ba0..3fae9462 100755 --- a/src/communicatorutils/connectnewpeers.py +++ b/src/communicatorutils/connectnewpeers.py @@ -1,9 +1,19 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Connect a new peer to our communicator instance. Does so randomly if no peer is specified -''' -''' +Connect a new peer to our communicator instance. +Does so randomly if no peer is specified +""" +import time +import secrets + +import onionrexceptions +import logger +import onionrpeers +from utils import networkmerger, gettransports +from onionrutils import stringvalidators, epoch +from communicator import peeraction, bootstrappeers +from coredb import keydb +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,13 +26,9 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import time, sys, secrets -import onionrexceptions, logger, onionrpeers -from utils import networkmerger, gettransports -from onionrutils import stringvalidators, epoch -from communicator import peeraction, bootstrappeers -from coredb import keydb +""" + + def connect_new_peer_to_communicator(comm_inst, peer='', useBootstrap=False): config = comm_inst.config retData = False @@ -32,14 +38,18 @@ def connect_new_peer_to_communicator(comm_inst, peer='', useBootstrap=False): if stringvalidators.validate_transport(peer): peerList = [peer] else: - raise onionrexceptions.InvalidAddress('Will not attempt connection test to invalid address') + raise onionrexceptions.InvalidAddress( + 'Will not attempt connection test to invalid address') else: peerList = keydb.listkeys.list_adders() mainPeerList = keydb.listkeys.list_adders() peerList = onionrpeers.get_score_sorted_peer_list() - # If we don't have enough peers connected or random chance, select new peers to try + """ + If we don't have enough peers connected or random chance, + select new peers to try + """ if len(peerList) < 8 or secrets.randbelow(4) == 3: tryingNew = [] for x in comm_inst.newPeers: @@ -60,8 +70,12 @@ def connect_new_peer_to_communicator(comm_inst, peer='', useBootstrap=False): # Don't connect to our own address if address in transports: continue - # Don't connect to invalid address or if its already been tried/connected, or if its cooled down - if len(address) == 0 or address in tried or address in comm_inst.onlinePeers or address in comm_inst.cooldownPeer: + """Don't connect to invalid address or + if its already been tried/connected, or if its cooled down + """ + if len(address) == 0 or address in tried \ + or address in comm_inst.onlinePeers \ + or address in comm_inst.cooldownPeer: continue if comm_inst.shutdown: return @@ -70,7 +84,7 @@ def connect_new_peer_to_communicator(comm_inst, peer='', useBootstrap=False): if ret == 'pong!': time.sleep(0.1) if address not in mainPeerList: - # Add a peer to our list if it isn't already since it successfully connected + # Add a peer to our list if it isn't already since it connected networkmerger.mergeAdders(address) if address not in comm_inst.onlinePeers: logger.info('Connected to ' + address, terminal=True) @@ -83,7 +97,8 @@ def connect_new_peer_to_communicator(comm_inst, peer='', useBootstrap=False): if profile.address == address: break else: - comm_inst.peerProfiles.append(onionrpeers.PeerProfiles(address)) + comm_inst.peerProfiles.append( + onionrpeers.PeerProfiles(address)) break else: # Mark a peer as tried if they failed to respond to ping diff --git a/src/communicatorutils/cooldownpeer.py b/src/communicatorutils/cooldownpeer.py index 6b797dde..d415c648 100755 --- a/src/communicatorutils/cooldownpeer.py +++ b/src/communicatorutils/cooldownpeer.py @@ -1,9 +1,10 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Select a random online peer in a communicator instance and have them "cool down" -''' -''' +Select random online peer in a communicator instance and have them "cool down" +""" +from onionrutils import epoch +from communicator import onlinepeers +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,39 +17,39 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -from onionrutils import epoch -from communicator import onlinepeers +""" + + def cooldown_peer(comm_inst): - '''Randomly add an online peer to cooldown, so we can connect a new one''' + """Randomly add an online peer to cooldown, so we can connect a new one.""" config = comm_inst.config - onlinePeerAmount = len(comm_inst.onlinePeers) + online_peer_amount = len(comm_inst.onlinePeers) minTime = 300 - cooldownTime = 600 - toCool = '' + cooldown_time = 600 + to_cool = '' tempConnectTimes = dict(comm_inst.connectTimes) # Remove peers from cooldown that have been there long enough tempCooldown = dict(comm_inst.cooldownPeer) for peer in tempCooldown: - if (epoch.get_epoch() - tempCooldown[peer]) >= cooldownTime: + if (epoch.get_epoch() - tempCooldown[peer]) >= cooldown_time: del comm_inst.cooldownPeer[peer] # Cool down a peer, if we have max connections alive for long enough - if onlinePeerAmount >= config.get('peers.max_connect', 10, save = True): + if online_peer_amount >= config.get('peers.max_connect', 10, save=True): finding = True while finding: try: - toCool = min(tempConnectTimes, key=tempConnectTimes.get) - if (epoch.get_epoch() - tempConnectTimes[toCool]) < minTime: - del tempConnectTimes[toCool] + to_cool = min(tempConnectTimes, key=tempConnectTimes.get) + if (epoch.get_epoch() - tempConnectTimes[to_cool]) < minTime: + del tempConnectTimes[to_cool] else: finding = False except ValueError: break else: - onlinepeers.remove_online_peer(comm_inst, toCool) - comm_inst.cooldownPeer[toCool] = epoch.get_epoch() + onlinepeers.remove_online_peer(comm_inst, to_cool) + comm_inst.cooldownPeer[to_cool] = epoch.get_epoch() - comm_inst.decrementThreadCount('cooldown_peer') \ No newline at end of file + comm_inst.decrementThreadCount('cooldown_peer') diff --git a/src/communicatorutils/deniableinserts.py b/src/communicatorutils/deniableinserts.py index 0ecc6d50..3252d3a3 100755 --- a/src/communicatorutils/deniableinserts.py +++ b/src/communicatorutils/deniableinserts.py @@ -1,9 +1,12 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Use the communicator to insert fake mail messages -''' -''' +Use the communicator to insert fake mail messages +""" +import secrets + +from etc import onionrvalues +import onionrblocks +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,17 +19,18 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import secrets -from etc import onionrvalues -import onionrblocks +""" + + def insert_deniable_block(comm_inst): - '''Insert a fake block in order to make it more difficult to track real blocks''' + """Insert a fake block to make it more difficult to track real blocks.""" fakePeer = '' chance = 10 if secrets.randbelow(chance) == (chance - 1): # This assumes on the libsodium primitives to have key-privacy fakePeer = onionrvalues.DENIABLE_PEER_ADDRESS data = secrets.token_hex(secrets.randbelow(5120) + 1) - onionrblocks.insert(data, header='pm', encryptType='asym', asymPeer=fakePeer, disableForward=True, meta={'subject': 'foo'}) - comm_inst.decrementThreadCount('insert_deniable_block') \ No newline at end of file + onionrblocks.insert(data, header='pm', encryptType='asym', + asymPeer=fakePeer, disableForward=True, + meta={'subject': 'foo'}) + comm_inst.decrementThreadCount('insert_deniable_block') diff --git a/src/communicatorutils/downloadblocks/__init__.py b/src/communicatorutils/downloadblocks/__init__.py index 30d62788..11c08ef4 100755 --- a/src/communicatorutils/downloadblocks/__init__.py +++ b/src/communicatorutils/downloadblocks/__init__.py @@ -126,7 +126,7 @@ def download_blocks_from_communicator(comm_inst: "OnionrCommunicatorDaemon"): f'/daemon-event/upload_event', post=True, is_json=True, - postData={'block': blockHash} + post_data={'block': blockHash} ) else: logger.warn('POW failed for block %s.' % (blockHash,)) diff --git a/src/communicatorutils/housekeeping.py b/src/communicatorutils/housekeeping.py index 950e2813..d2cc870d 100755 --- a/src/communicatorutils/housekeeping.py +++ b/src/communicatorutils/housekeeping.py @@ -4,6 +4,7 @@ Cleanup old Onionr blocks and forward secrecy keys using the communicator. Ran from a communicator timer usually """ import sqlite3 + import logger from onionrusers import onionrusers from onionrutils import epoch @@ -67,7 +68,8 @@ def clean_keys(comm_inst): time = epoch.get_epoch() deleteKeys = [] - for entry in c.execute("SELECT * FROM forwardKeys WHERE expire <= ?", (time,)): + for entry in c.execute( + "SELECT * FROM forwardKeys WHERE expire <= ?", (time,)): logger.debug('Forward key: %s' % entry[1]) deleteKeys.append(entry[1]) diff --git a/src/communicatorutils/lookupadders.py b/src/communicatorutils/lookupadders.py index 1f042b0b..9dfce1cb 100755 --- a/src/communicatorutils/lookupadders.py +++ b/src/communicatorutils/lookupadders.py @@ -1,9 +1,14 @@ -''' - Onionr - Private P2P Communication +""" +Onionr - Private P2P Communication. - Lookup new peer transport addresses using the communicator -''' -''' +Lookup new peer transport addresses using the communicator +""" +import logger +from onionrutils import stringvalidators +from communicator import peeraction, onlinepeers +from utils import gettransports +import onionrexceptions +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,12 +21,9 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import logger -from onionrutils import stringvalidators -from communicator import peeraction, onlinepeers -from utils import gettransports -import onionrexceptions +""" + + def lookup_new_peer_transports_with_communicator(comm_inst): logger.info('Looking up new addresses...') tryAmount = 1 @@ -47,7 +49,8 @@ def lookup_new_peer_transports_with_communicator(comm_inst): invalid = [] for x in newPeers: x = x.strip() - if not stringvalidators.validate_transport(x) or x in comm_inst.newPeers or x in transports: + if not stringvalidators.validate_transport(x) \ + or x in comm_inst.newPeers or x in transports: # avoid adding if its our address invalid.append(x) for x in invalid: @@ -56,4 +59,5 @@ def lookup_new_peer_transports_with_communicator(comm_inst): except ValueError: pass comm_inst.newPeers.extend(newPeers) - comm_inst.decrementThreadCount('lookup_new_peer_transports_with_communicator') \ No newline at end of file + comm_inst.decrementThreadCount( + 'lookup_new_peer_transports_with_communicator') diff --git a/src/communicatorutils/lookupblocks.py b/src/communicatorutils/lookupblocks.py index 873863d1..d97be65b 100755 --- a/src/communicatorutils/lookupblocks.py +++ b/src/communicatorutils/lookupblocks.py @@ -1,4 +1,4 @@ -"""Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. Lookup new blocks with the communicator using a random connected peer """ @@ -8,7 +8,7 @@ import logger import onionrproofs from onionrutils import stringvalidators, epoch from communicator import peeraction, onlinepeers -from coredb import blockmetadb +from coredb.blockmetadb import get_block_list from utils import reconstructhash from onionrblocks import onionrblacklist import onionrexceptions @@ -36,20 +36,24 @@ def lookup_blocks_from_communicator(comm_inst): logger.info('Looking up new blocks') tryAmount = 2 newBlocks = '' - existingBlocks = blockmetadb.get_block_list() # List of existing saved blocks - triedPeers = [] # list of peers we've tried this time around - maxBacklog = 1560 # Max amount of *new* block hashes to have already in queue, to avoid memory exhaustion - lastLookupTime = 0 # Last time we looked up a particular peer's list + # List of existing saved blocks + existingBlocks = get_block_list() + triedPeers = [] # list of peers we've tried this time around + # Max amount of *new* block hashes to have in queue + maxBacklog = 1560 + lastLookupTime = 0 # Last time we looked up a particular peer's list new_block_count = 0 for i in range(tryAmount): - listLookupCommand = 'getblocklist' # This is defined here to reset it each time + # Defined here to reset it each time, time offset is added later + listLookupCommand = 'getblocklist' if len(comm_inst.blockQueue) >= maxBacklog: break if not comm_inst.isOnline: break # check if disk allocation is used if comm_inst.storage_counter.is_full(): - logger.debug('Not looking up new blocks due to maximum amount of allowed disk space used') + logger.debug( + 'Not looking up new blocks due to maximum amount of disk used') break try: # select random online peer @@ -65,33 +69,44 @@ def lookup_blocks_from_communicator(comm_inst): continue triedPeers.append(peer) - # Get the last time we looked up a peer's stamp to only fetch blocks since then. + # Get the last time we looked up a peer's stamp, + # to only fetch blocks since then. # Saved in memory only for privacy reasons try: lastLookupTime = comm_inst.dbTimestamps[peer] except KeyError: - lastLookupTime = epoch.get_epoch() - config.get("general.max_block_age", onionrvalues.DEFAULT_EXPIRE) + lastLookupTime = epoch.get_epoch() - \ + config.get("general.max_block_age", + onionrvalues.DEFAULT_EXPIRE) listLookupCommand += '?date=%s' % (lastLookupTime,) try: - newBlocks = peeraction.peer_action(comm_inst, peer, listLookupCommand) # get list of new block hashes + newBlocks = peeraction.peer_action( + comm_inst, + peer, listLookupCommand) # get list of new block hashes except Exception as error: - logger.warn('Could not get new blocks from %s.' % peer, error = error) + logger.warn( + f'Could not get new blocks from {peer}.', + error=error) newBlocks = False - else: - comm_inst.dbTimestamps[peer] = epoch.get_rounded_epoch(roundS=60) - if newBlocks != False: + + if newBlocks != False: # noqa # if request was a success for i in newBlocks.split('\n'): if stringvalidators.validate_hash(i): i = reconstructhash.reconstruct_hash(i) # if newline seperated string is valid hash - if not i in existingBlocks: - # if block does not exist on disk and is not already in block queue + + # if block does not exist on disk + is not already in queue + if i not in existingBlocks: if i not in comm_inst.blockQueue: - if onionrproofs.hashMeetsDifficulty(i) and not blacklist.inBlacklist(i): + if onionrproofs.hashMeetsDifficulty(i) and \ + not blacklist.inBlacklist(i): if len(comm_inst.blockQueue) <= 1000000: - comm_inst.blockQueue[i] = [peer] # add blocks to download queue + # add blocks to download queue + comm_inst.blockQueue[i] = [peer] new_block_count += 1 + comm_inst.dbTimestamps[peer] = \ + epoch.get_rounded_epoch(roundS=60) else: if peer not in comm_inst.blockQueue[i]: if len(comm_inst.blockQueue[i]) < 10: @@ -100,7 +115,9 @@ def lookup_blocks_from_communicator(comm_inst): block_string = "" if new_block_count > 1: block_string = "s" - logger.info('Discovered %s new block%s' % (new_block_count, block_string), terminal=True) - comm_inst.download_blocks_timer.count = int(comm_inst.download_blocks_timer.frequency * 0.99) + logger.info( + f'Discovered {new_block_count} new block{block_string}', + terminal=True) + comm_inst.download_blocks_timer.count = \ + int(comm_inst.download_blocks_timer.frequency * 0.99) comm_inst.decrementThreadCount('lookup_blocks_from_communicator') - return diff --git a/src/communicatorutils/netcheck.py b/src/communicatorutils/netcheck.py index 415c2d1e..e3ad4870 100755 --- a/src/communicatorutils/netcheck.py +++ b/src/communicatorutils/netcheck.py @@ -1,9 +1,9 @@ """ - Onionr - Private P2P Communication +Onionr - Private P2P Communication. - Determine if our node is able to use Tor based - on the status of a communicator instance - and the result of pinging onion http servers +Determine if our node is able to use Tor based +on the status of a communicator instance +and the result of pinging onion http servers """ import logger from utils import netutils @@ -26,8 +26,10 @@ from . import restarttor def net_check(comm_inst): - """Check if we are connected to the internet - or not when we can't connect to any peers""" + """Check if we are connected to the internet. + + or not when we can't connect to any peers + """ # for detecting if we have received incoming connections recently rec = False if len(comm_inst.onlinePeers) == 0: @@ -42,7 +44,8 @@ def net_check(comm_inst): if not comm_inst.shutdown: if not comm_inst.config.get('general.offline_mode', False): logger.warn('Network check failed, are you connected to ' + - 'the Internet, and is Tor working?', + 'the Internet, and is Tor working? ' + + 'This is usually temporary, but bugs and censorship can cause this to persist, in which case you should report it to beardog [at] mailbox.org', # noqa terminal=True) restarttor.restart(comm_inst) comm_inst.offlinePeers = [] diff --git a/src/communicatorutils/onionrcommunicatortimers.py b/src/communicatorutils/onionrcommunicatortimers.py index 2a464bd7..c0d8b1ca 100755 --- a/src/communicatorutils/onionrcommunicatortimers.py +++ b/src/communicatorutils/onionrcommunicatortimers.py @@ -1,10 +1,20 @@ -''' - Onionr - Private P2P Communication +""" +Onionr - Private P2P Communication. - This file contains timer control for the communicator -''' +This file contains timer control for the communicator +""" from __future__ import annotations # thank you python, very cool -''' +import uuid +import threading + +import onionrexceptions +import logger + +from typing import TYPE_CHECKING +from typing import Callable, NewType, Iterable +if TYPE_CHECKING: + from communicator import OnionrCommunicatorDaemon +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -17,26 +27,18 @@ from __future__ import annotations # thank you python, very cool You should have received a copy of the GNU General Public License along with this program. If not, see . -''' +""" -import uuid -import threading - -import onionrexceptions, logger - -from typing import TYPE_CHECKING -from typing import Callable, NewType, Iterable -from psutil import Process -if TYPE_CHECKING: - from communicator import OnionrCommunicatorDaemon CallFreqSeconds = NewType('CallFreqSeconds', int) + class OnionrCommunicatorTimers: def __init__(self, daemon_inst: OnionrCommunicatorDaemon, - timer_function: Callable, frequency: CallFreqSeconds, - make_thread:bool=True, thread_amount:int=1, max_threads:int=5, - requires_peer:bool=False, my_args:Iterable=[]): + timer_function: Callable, frequency: CallFreqSeconds, + make_thread: bool = True, + thread_amount: int = 1, max_threads: int = 5, + requires_peer: bool = False, my_args: Iterable = []): self.timer_function = timer_function self.frequency = frequency self.thread_amount = thread_amount @@ -51,30 +53,44 @@ class OnionrCommunicatorTimers: def processTimer(self): - # mark how many instances of a thread we have (decremented at thread end) + # mark # of instances of a thread we have (decremented at thread end) try: self.daemon_inst.threadCounts[self.timer_function.__name__] except KeyError: self.daemon_inst.threadCounts[self.timer_function.__name__] = 0 - # execute thread if it is time, and we are not missing *required* online peer + # execute timer's func, if we are not missing *required* online peer if self.count == self.frequency and not self.daemon_inst.shutdown: try: - if self.requires_peer and len(self.daemon_inst.onlinePeers) == 0: + if self.requires_peer and \ + len(self.daemon_inst.onlinePeers) == 0: raise onionrexceptions.OnlinePeerNeeded except onionrexceptions.OnlinePeerNeeded: return else: if self.make_thread: for i in range(self.thread_amount): - if self.daemon_inst.threadCounts[self.timer_function.__name__] >= self.max_threads: - logger.debug('%s is currently using the maximum number of threads, not starting another.' % self.timer_function.__name__) + """ + Log if a timer has max num of active threads + If this logs frequently it is indicative of a bug + or need for optimization + """ + if self.daemon_inst.threadCounts[ + self.timer_function.__name__] >= \ + self.max_threads: + logger.debug( + f'{self.timer_function.__name__} is currently using the maximum number of threads, not starting another.') # noqa + # if active number of threads for timer not reached yet else: - self.daemon_inst.threadCounts[self.timer_function.__name__] += 1 - newThread = threading.Thread(target=self.timer_function, args=self.args, daemon=True, - name=self.timer_function.__name__ + ' - ' + str(uuid.uuid4())) + self.daemon_inst.threadCounts[ + self.timer_function.__name__] += 1 + newThread = threading.Thread( + target=self.timer_function, args=self.args, + daemon=True, + name=self.timer_function.__name__ + ' - ' + + str(uuid.uuid4())) newThread.start() else: self.timer_function() - self.count = -1 # negative 1 because its incremented at bottom + self.count = -1 # negative 1 because its incremented at bottom self.count += 1 diff --git a/src/communicatorutils/proxypicker.py b/src/communicatorutils/proxypicker.py index c8981dae..85dc4d5b 100755 --- a/src/communicatorutils/proxypicker.py +++ b/src/communicatorutils/proxypicker.py @@ -1,9 +1,9 @@ -''' - Onionr - Private P2P Communication +""" +Onionr - Private P2P Communication. - Just picks a proxy to use based on a peer's address -''' -''' +Pick a proxy to use based on a peer's address +""" +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,11 +16,13 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' +""" + def pick_proxy(peer_address): if peer_address.endswith('.onion'): return 'tor' elif peer_address.endswith('.i2p'): return 'i2p' - raise ValueError(f"Peer address was not string ending with acceptable value: {peer_address}") \ No newline at end of file + raise ValueError( + f"Peer address not ending with acceptable domain: {peer_address}") diff --git a/src/communicatorutils/restarttor.py b/src/communicatorutils/restarttor.py index 232dd57b..9ffde83d 100644 --- a/src/communicatorutils/restarttor.py +++ b/src/communicatorutils/restarttor.py @@ -1,5 +1,25 @@ +""" +Onionr - Private P2P Communication. + +Restart Onionr managed Tor +""" import netcontroller import config +""" + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +""" + def restart(comm_inst): if not config.get('tor.use_existing_tor', False): diff --git a/src/communicatorutils/servicecreator.py b/src/communicatorutils/servicecreator.py index 9ff031d9..52228f5f 100755 --- a/src/communicatorutils/servicecreator.py +++ b/src/communicatorutils/servicecreator.py @@ -1,9 +1,14 @@ -''' - Onionr - Private P2P Communication +""" +Onionr - Private P2P Communication. - Creates an onionr direct connection service by scanning all connection blocks -''' -''' +Creates an onionr direct connection service by scanning all connection blocks +""" +import communicator +from onionrblocks import onionrblockapi +from onionrutils import stringvalidators, bytesconverter +from coredb import blockmetadb +from onionrservices import server_exists +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,26 +21,23 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import communicator -from onionrblocks import onionrblockapi -import logger -from onionrutils import stringvalidators, bytesconverter -from coredb import blockmetadb -from onionrservices import server_exists +""" + + def service_creator(daemon): assert isinstance(daemon, communicator.OnionrCommunicatorDaemon) - + # Find socket connection blocks # TODO cache blocks and only look at recently received ones con_blocks = blockmetadb.get_blocks_by_type('con') for b in con_blocks: - if not b in daemon.active_services: + if b not in daemon.active_services: bl = onionrblockapi.Block(b, decrypt=True) bs = bytesconverter.bytes_to_str(bl.bcontent) + '.onion' if server_exists(bl.signer): continue - if stringvalidators.validate_pub_key(bl.signer) and stringvalidators.validate_transport(bs): + if stringvalidators.validate_pub_key(bl.signer) and \ + stringvalidators.validate_transport(bs): signer = bytesconverter.bytes_to_str(bl.signer) daemon.active_services.append(b) daemon.active_services.append(signer) diff --git a/src/config/__init__.py b/src/config/__init__.py index 8f2ede60..b292c4be 100755 --- a/src/config/__init__.py +++ b/src/config/__init__.py @@ -2,7 +2,11 @@ This file deals with configuration management. """ -import os, json, logger +import os +from json import JSONDecodeError + +import ujson as json +import logger import filepaths from . import onboarding @@ -102,7 +106,7 @@ def save(): try: with open(get_config_file(), 'w', encoding="utf8") as configfile: json.dump(get_config(), configfile, indent=2) - except json.JSONDecodeError: + except JSONDecodeError: logger.warn('Failed to write to configuration file.') @@ -112,7 +116,7 @@ def reload(): try: with open(get_config_file(), 'r', encoding="utf8") as configfile: set_config(json.loads(configfile.read())) - except (FileNotFoundError, json.JSONDecodeError) as e: + except (FileNotFoundError, JSONDecodeError) as e: pass #logger.debug('Failed to parse configuration file.') diff --git a/src/config/onboarding.py b/src/config/onboarding.py index e439d09f..b090b60d 100644 --- a/src/config/onboarding.py +++ b/src/config/onboarding.py @@ -43,6 +43,7 @@ def set_config_from_onboarding(config_settings: OnboardingConfig): if get(config_settings, 'localThreat'): config.set('general.security_level', 3) + config.set('transports.lan', False) config.set('ui.theme', 'light') if get(config_settings, 'useDark'): diff --git a/src/etc/onionrvalues.py b/src/etc/onionrvalues.py index 05825205..bbf2f8e5 100755 --- a/src/etc/onionrvalues.py +++ b/src/etc/onionrvalues.py @@ -23,7 +23,7 @@ import filepaths DENIABLE_PEER_ADDRESS = "OVPCZLOXD6DC5JHX4EQ3PSOGAZ3T24F75HQLIUZSDSMYPEOXCPFA" PASSWORD_LENGTH = 25 ONIONR_TAGLINE = 'Private P2P Communication - GPLv3 - https://Onionr.net' -ONIONR_VERSION = '3.1.0' +ONIONR_VERSION = '4.0.0' ONIONR_VERSION_CODENAME = 'Genesis' ONIONR_VERSION_TUPLE = tuple(ONIONR_VERSION.split('.')) # (MAJOR, MINOR, VERSION) API_VERSION = '1' # increments of 1; only change when something fundamental about how the API works changes. This way other nodes know how to communicate without learning too much information about you. diff --git a/src/filepaths/__init__.py b/src/filepaths/__init__.py index 2870f859..a929518d 100644 --- a/src/filepaths/__init__.py +++ b/src/filepaths/__init__.py @@ -36,3 +36,5 @@ onboarding_mark_file = home + 'onboarding-completed' log_file = home + 'onionr.log' ephemeral_services_file = home + 'ephemeral-services.list' + +restarting_indicator = home + "is-restarting" diff --git a/src/httpapi/apiutils/getblockdata.py b/src/httpapi/apiutils/getblockdata.py index ad420ae1..fc9a615d 100644 --- a/src/httpapi/apiutils/getblockdata.py +++ b/src/httpapi/apiutils/getblockdata.py @@ -1,4 +1,5 @@ -import json +import ujson as json + from onionrblocks import onionrblockapi from onionrutils import bytesconverter, stringvalidators import onionrexceptions diff --git a/src/httpapi/configapi/__init__.py b/src/httpapi/configapi/__init__.py index e1226a3a..a0aefab7 100755 --- a/src/httpapi/configapi/__init__.py +++ b/src/httpapi/configapi/__init__.py @@ -17,8 +17,10 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . """ -import json +from json import JSONDecodeError +import ujson as json from flask import Blueprint, request, Response, abort + import config, onionrutils from onionrutils.bytesconverter import bytes_to_str @@ -41,7 +43,7 @@ def set_all_config(): """Overwrite existing JSON config with new JSON string""" try: new_config = request.get_json(force=True) - except json.JSONDecodeError: + except JSONDecodeError: abort(400) else: config.set_config(new_config) @@ -58,7 +60,7 @@ def set_by_key(key): """ try: data = json.loads(bytes_to_str(request.data)) - except (json.JSONDecodeError, KeyError): + except (JSONDecodeError, KeyError): abort(400) config.set(key, data, True) return Response('success') \ No newline at end of file diff --git a/src/httpapi/friendsapi/__init__.py b/src/httpapi/friendsapi/__init__.py index 4447cc9c..03bc552c 100755 --- a/src/httpapi/friendsapi/__init__.py +++ b/src/httpapi/friendsapi/__init__.py @@ -1,9 +1,13 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - This file creates http endpoints for friend management -''' -''' +This file creates http endpoints for friend management +""" +import ujson as json + +from onionrusers import contactmanager +from flask import Blueprint, Response, request, abort, redirect +from coredb import keydb +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,12 +20,7 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import json -from onionrusers import contactmanager -from flask import Blueprint, Response, request, abort, redirect -from coredb import keydb - +""" friends = Blueprint('friends', __name__) @friends.route('/friends/list') diff --git a/src/httpapi/insertblock.py b/src/httpapi/insertblock.py index c245ea87..ce25b968 100644 --- a/src/httpapi/insertblock.py +++ b/src/httpapi/insertblock.py @@ -2,7 +2,7 @@ Create blocks with the client api server """ -import json +import ujson as json import threading from flask import Blueprint, Response, request, g diff --git a/src/httpapi/miscclientapi/endpoints.py b/src/httpapi/miscclientapi/endpoints.py index 0c48dfea..a9439a6f 100644 --- a/src/httpapi/miscclientapi/endpoints.py +++ b/src/httpapi/miscclientapi/endpoints.py @@ -8,7 +8,6 @@ import platform from flask import Response, Blueprint, request, send_from_directory, abort from flask import g -from gevent import spawn import unpaddedbase32 from httpapi import apiutils @@ -20,6 +19,7 @@ from onionrutils import mnemonickeys from onionrutils import bytesconverter from etc import onionrvalues from utils import reconstructhash +from utils.gettransports import get as get_tor """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -40,6 +40,7 @@ pub_key = onionrcrypto.pub_key.replace('=', '') SCRIPT_NAME = os.path.dirname(os.path.realpath(__file__)) + \ f'/../../../{onionrvalues.SCRIPT_NAME}' + class PrivateEndpoints: def __init__(self, client_api): private_endpoints_bp = Blueprint('privateendpoints', __name__) @@ -74,7 +75,6 @@ class PrivateEndpoints: raise ValueError('block hash needs to be alpha numeric') name = reconstructhash.reconstruct_hash(name) if name in client_api.publicAPI.hideBlocks: - spawn(_delay_wait_for_share_block_removal) return Response("will be removed") else: client_api.publicAPI.hideBlocks.append(name) @@ -140,3 +140,10 @@ class PrivateEndpoints: def is_tor_ready(): """If Tor is starting up, the web UI is not ready to be used.""" return Response(str(g.too_many.get(NetController).readyState).lower()) + + @private_endpoints_bp.route('/gettoraddress') + def get_tor_address(): + """Return public Tor v3 Onion address for this node""" + if not config.get('general.security_level', 0) == 0: + abort(404) + return Response(get_tor()[0]) diff --git a/src/httpapi/miscpublicapi/getblocks.py b/src/httpapi/miscpublicapi/getblocks.py index 887e8661..55b8aba4 100755 --- a/src/httpapi/miscpublicapi/getblocks.py +++ b/src/httpapi/miscpublicapi/getblocks.py @@ -1,9 +1,16 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Public endpoints to get block data and lists -''' -''' +Public endpoints to get block data and lists +""" +from flask import Response, abort + +import config +from onionrutils import bytesconverter, stringvalidators +from coredb import blockmetadb +from utils import reconstructhash +from onionrblocks import BlockList +from .. import apiutils +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,13 +23,9 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -from flask import Response, abort -import config -from onionrutils import bytesconverter, stringvalidators -from coredb import blockmetadb -from utils import reconstructhash -from .. import apiutils +""" + + def get_public_block_list(publicAPI, request): # Provide a list of our blocks, with a date offset dateAdjust = request.args.get('date') @@ -37,15 +40,16 @@ def get_public_block_list(publicAPI, request): share_list += '%s\n' % (reconstructhash.deconstruct_hash(b),) return Response(share_list) + def get_block_data(publicAPI, data): - '''data is the block hash in hex''' + """data is the block hash in hex""" resp = '' if stringvalidators.validate_hash(data): if not config.get('general.hide_created_blocks', True) or data not in publicAPI.hideBlocks: - if data in blockmetadb.get_block_list(): + if data in publicAPI._too_many.get(BlockList).get(): block = apiutils.GetBlockData().get_block_data(data, raw=True, decrypt=False) try: - block = block.encode() # Encode in case data is binary + block = block.encode('utf-8') # Encode in case data is binary except AttributeError: if len(block) == 0: abort(404) diff --git a/src/httpapi/miscpublicapi/upload.py b/src/httpapi/miscpublicapi/upload.py index 3a020f07..6a4e102a 100755 --- a/src/httpapi/miscpublicapi/upload.py +++ b/src/httpapi/miscpublicapi/upload.py @@ -1,21 +1,21 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. + +Accept block uploads to the public API server +""" +import sys - Accept block uploads to the public API server -''' from gevent import spawn from gevent import threading - -import sys from flask import Response from flask import abort +from flask import g from onionrutils import localcommand from onionrblocks import blockimporter import onionrexceptions import logger -''' +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -28,7 +28,7 @@ import logger You should have received a copy of the GNU General Public License along with this program. If not, see . -''' +""" def accept_upload(request): @@ -40,17 +40,19 @@ def accept_upload(request): try: b_hash = blockimporter.import_block_from_data(data) if b_hash: - spawn( - localcommand.local_command, - f'/daemon-event/upload_event', - post=True, - is_json=True, - postData={'block': b_hash} - ).get(timeout=10) + if g.too_many.get_by_string("OnionrCommunicatorDaemon").onlinePeers: + spawn( + localcommand.local_command, + f'/daemon-event/upload_event', + post=True, + is_json=True, + post_data={'block': b_hash} + ).get(timeout=10) resp = 'success' else: resp = 'failure' - logger.warn('Error encountered importing uploaded block') + logger.warn( + f'Error encountered importing uploaded block {b_hash}') except onionrexceptions.BlacklistedBlock: logger.debug('uploaded block is blacklisted') resp = 'failure' @@ -62,6 +64,8 @@ def accept_upload(request): abort(400) elif resp == 'proof': resp = Response(resp, 400) + logger.warn( + f'Error importing uploaded block, invalid proof {b_hash}') else: resp = Response(resp) return resp diff --git a/src/httpapi/onionrsitesapi/__init__.py b/src/httpapi/onionrsitesapi/__init__.py index 0cfe0fec..9526aa76 100644 --- a/src/httpapi/onionrsitesapi/__init__.py +++ b/src/httpapi/onionrsitesapi/__init__.py @@ -1,21 +1,6 @@ -""" - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - view and interact with onionr sites -""" -""" - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . +view and interact with onionr sites """ import base64 import binascii @@ -30,6 +15,21 @@ import onionrexceptions from onionrutils import stringvalidators from onionrutils import mnemonickeys from . import sitefiles +""" + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +""" + site_api = Blueprint('siteapi', __name__) diff --git a/src/httpapi/onionrsitesapi/sitefiles.py b/src/httpapi/onionrsitesapi/sitefiles.py index 79d66d49..f9171926 100644 --- a/src/httpapi/onionrsitesapi/sitefiles.py +++ b/src/httpapi/onionrsitesapi/sitefiles.py @@ -1,21 +1,6 @@ -""" - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Read onionr site files -""" -""" - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . +Read onionr site files """ from typing import Union, Tuple import tarfile @@ -32,30 +17,52 @@ from onionrblocks import insert from onionrtypes import UserID, DeterministicKeyPassphrase, BlockHash from onionrcrypto import generate_deterministic +""" + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . +""" + def find_site_gzip(user_id: str)->tarfile.TarFile: """Return verified site tar object""" sites = blockmetadb.get_blocks_by_type('osite') user_site = None + unpadded_user = user_id user_id = unpaddedbase32.repad(user_id) for site in sites: block = onionrblockapi.Block(site) - if block.isSigner(user_id): + if block.isSigner(user_id) or block.isSigner(unpadded_user): user_site = block if not user_site is None: return tarfile.open(fileobj=io.BytesIO(user_site.bcontent), mode='r') return None + def get_file(user_id, file)->Union[bytes, None]: """Get a site file content""" ret_data = "" site = find_site_gzip(user_id) + + if file.endswith('/'): + file += 'index.html' if site is None: return None for t_file in site.getmembers(): + if t_file.name.replace('./', '') == file: return site.extractfile(t_file) return None + def create_site(admin_pass: DeterministicKeyPassphrase, directory:str='.')->Tuple[UserID, BlockHash]: public_key, private_key = generate_deterministic(admin_pass) diff --git a/src/httpapi/security/client.py b/src/httpapi/security/client.py index fbb6b288..77cb5d06 100644 --- a/src/httpapi/security/client.py +++ b/src/httpapi/security/client.py @@ -1,9 +1,12 @@ -''' - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Process incoming requests to the client api server to validate they are legitimate -''' -''' +Process incoming requests to the client api server to validate they are legitimate +""" +import hmac +from flask import Blueprint, request, abort, g +from onionrservices import httpheaders +from . import pluginwhitelist +""" This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or @@ -16,11 +19,7 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . -''' -import hmac -from flask import Blueprint, request, abort, g -from onionrservices import httpheaders -from . import pluginwhitelist +""" # Be extremely mindful of this. These are endpoints available without a password whitelist_endpoints = ['www', 'staticfiles.homedata', 'staticfiles.sharedContent', @@ -36,7 +35,7 @@ class ClientAPISecurity: @client_api_security_bp.before_app_request def validate_request(): - '''Validate request has set password and is the correct hostname''' + """Validate request has set password and is the correct hostname""" # For the purpose of preventing DNS rebinding attacks if request.host != '%s:%s' % (client_api.host, client_api.bindPort): abort(403) @@ -66,5 +65,5 @@ class ClientAPISecurity: if request.endpoint in ('siteapi.site', 'siteapi.siteFile'): resp.headers['Content-Security-Policy'] = "default-src 'none'; style-src 'self' data: 'unsafe-inline'; img-src 'self' data:; media-src 'self' data:" else: - resp.headers['Content-Security-Policy'] = "default-src 'none'; script-src 'self'; object-src 'none'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; media-src 'none'; frame-src 'none'; font-src 'self'; connect-src 'self'" + resp.headers['Content-Security-Policy'] = "default-src 'none'; script-src 'self'; object-src 'none'; style-src 'self' 'unsafe-inline'; img-src 'self' data:; media-src 'self'; frame-src 'none'; font-src 'self'; connect-src 'self'" return resp diff --git a/src/httpapi/security/pluginwhitelist.py b/src/httpapi/security/pluginwhitelist.py index de6c8baa..ecfa280f 100644 --- a/src/httpapi/security/pluginwhitelist.py +++ b/src/httpapi/security/pluginwhitelist.py @@ -1,8 +1,8 @@ -""" - Onionr - Private P2P Communication +"""Onionr - Private P2P Communication. - Load web UI client endpoints into the whitelist from plugins +Load web UI client endpoints into the whitelist from plugins """ +import onionrplugins """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -17,7 +17,8 @@ You should have received a copy of the GNU General Public License along with this program. If not, see . """ -import onionrplugins + + def load_plugin_security_whitelist_endpoints(whitelist: list): """Accept a list reference of whitelist endpoints from security/client.py and append plugin's specified endpoints to them by attribute""" diff --git a/src/httpapi/security/public.py b/src/httpapi/security/public.py index a9d12f93..eaddf029 100644 --- a/src/httpapi/security/public.py +++ b/src/httpapi/security/public.py @@ -32,9 +32,12 @@ class PublicAPISecurity: """Validate request has the correct hostname""" # If high security level, deny requests to public # (HS should be disabled anyway for Tor, but might not be for I2P) + + g.is_onionr_client = False transports = gettransports.get() if public_api.config.get('general.security_level', default=1) > 0: abort(403) + if request.host not in transports: # Abort conn if wrong HTTP hostname, to prevent DNS rebinding abort(403) @@ -46,6 +49,11 @@ class PublicAPISecurity: g.is_onionr_client = False except KeyError: g.is_onionr_client = False + # Add shared objects + try: + g.too_many = public_api._too_many + except KeyError: + g.too_many = None @public_api_security_bp.after_app_request def send_headers(resp): @@ -57,10 +65,12 @@ class PublicAPISecurity: NON_NETWORK_HEADERS = ('Content-Security-Policy', 'X-Frame-Options', 'X-Content-Type-Options', 'Feature-Policy', 'Clear-Site-Data', 'Referrer-Policy') + try: if g.is_onionr_client: for header in NON_NETWORK_HEADERS: del resp.headers[header] except AttributeError: abort(403) + public_api.lastRequest = epoch.get_rounded_epoch(roundS=5) return resp diff --git a/src/httpapi/sse/README.md b/src/httpapi/sse/README.md index b01ca67a..9f3f6a41 100644 --- a/src/httpapi/sse/README.md +++ b/src/httpapi/sse/README.md @@ -1,3 +1,6 @@ # sse -This folder contains a wrapper for handling server sent event loops \ No newline at end of file +This folder contains a wrapper for handling server sent event loops + + + diff --git a/src/httpapi/sse/private/__init__.py b/src/httpapi/sse/private/__init__.py index 32934772..4213fd4f 100644 --- a/src/httpapi/sse/private/__init__.py +++ b/src/httpapi/sse/private/__init__.py @@ -2,10 +2,17 @@ SSE API for node client access """ +from pathlib import Path + from flask import g, Blueprint from gevent import sleep import gevent +import ujson +from onionrblocks.onionrblockapi import Block +from coredb.dbfiles import block_meta_db +from coredb.blockmetadb import get_block_list +from onionrutils.epoch import get_epoch from onionrstatistics.transports.tor import TorStats from .. import wrapper """ @@ -45,3 +52,27 @@ def stream_tor_circuits(): yield "data: " + tor_stats.get_json() + "\n\n" sleep(10) return SSEWrapper.handle_sse_request(circuit_stat_stream) + +@private_sse_blueprint.route('/recentblocks') +def stream_recent_blocks(): + def _compile_json(b_list): + js = {} + block_obj = None + for block in b_list: + block_obj = Block(block) + if block_obj.isEncrypted: + js[block] = 'encrypted' + else: + js[block] = Block(block).btype + return ujson.dumps({"blocks": js}, reject_bytes=True) + + def _stream_recent(): + last_time = Path(block_meta_db).stat().st_ctime + while True: + if Path(block_meta_db).stat().st_ctime != last_time: + last_time = Path(block_meta_db).stat().st_ctime + yield "data: " + _compile_json(get_block_list(get_epoch() - 5)) + "\n\n" + else: + yield "data: none" + "\n\n" + sleep(5) + return SSEWrapper.handle_sse_request(_stream_recent) diff --git a/src/lan/__init__.py b/src/lan/__init__.py index 230312df..f900a889 100644 --- a/src/lan/__init__.py +++ b/src/lan/__init__.py @@ -7,7 +7,6 @@ from threading import Thread if TYPE_CHECKING: from toomanyobjs import TooMany -from .client import Client from .discover import learn_services, advertise_service """ This program is free software: you can redistribute it and/or modify @@ -34,7 +33,6 @@ class LANManager: def start(self): - Thread(target=learn_services, args=[self.too_many.get(Client)], daemon=True).start() + Thread(target=learn_services, daemon=True).start() Thread(target=advertise_service, daemon=True).start() - Thread(target=self.too_many.get(Client, (self.peers,)).start, daemon=True).start() diff --git a/src/lan/client/__init__.py b/src/lan/client/__init__.py index d7f92f48..e3f881b1 100644 --- a/src/lan/client/__init__.py +++ b/src/lan/client/__init__.py @@ -2,11 +2,18 @@ LAN transport client thread """ -from typing import List +import requests -from onionrcrypto.cryptoutils.randomshuffle import random_shuffle +from typing import Set + +from onionrtypes import LANIP from utils.bettersleep import better_sleep -from onionrutils.basicrequests import do_post_request, do_get_request +import logger +from coredb.blockmetadb import get_block_list +from onionrblocks.blockimporter import import_block_from_data +from ..server import ports + +from threading import Thread """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -22,41 +29,39 @@ from onionrutils.basicrequests import do_post_request, do_get_request along with this program. If not, see . """ - -class ConnectedError(Exception): pass +connected_lan_peers: Set[LANIP] = set([]) +def _lan_work(peer: LANIP): + def _sync_peer(url): + our_blocks = get_block_list() + blocks = requests.get(url + 'blist/0').text.splitlines() + for block in blocks: + if block not in our_blocks: + import_block_from_data(requests.get(url + f'get/{block}', stream=True).raw.read(6000000)) -class Client: - def __init__(self): - self.peers = [] - self.lookup_time = {} - self.poll_delay = 10 - self.active_threads: set = set([]) - def get_lookup_time(self, peer): + for port in ports: try: - return self.lookup_time[peer] - except KeyError: - return 0 - - def peer_thread(self, peer): - def do_peer_sync(): return - if peer in self.active_threads: - raise ConnectedError - self.active_threads.add(peer) - do_peer_sync() - self.active_threads.remove(peer) - - def start(self): - while True: - peers = random_shuffle(list(set(self.peers) ^ self.active_threads)) - try: - self.peer_thread(peers[0]) - except IndexError: - pass + root = f'http://{peer}:{port}/' + if requests.get(f'{root}ping').text != 'onionr!': + connected_lan_peers.remove(peer) + else: + logger.info(f'[LAN] Connected to {peer}:{port}', terminal=True) + while True: + try: + _sync_peer(root) + except requests.exceptions.ConnectionError: + break + break + except requests.exceptions.ConnectionError: + pass + else: + connected_lan_peers.remove(peer) - - better_sleep(self.poll_delay) +def connect_peer(peer: LANIP): + if peer not in connected_lan_peers: + connected_lan_peers.add(peer) + Thread(target=_lan_work, args=[peer], daemon=True).start() diff --git a/src/lan/discover.py b/src/lan/discover.py index 3f6ed819..52ac1b46 100644 --- a/src/lan/discover.py +++ b/src/lan/discover.py @@ -9,8 +9,9 @@ from typing import List from ipaddress import ip_address from socket import SHUT_RDWR -from .getip import lan_ips +from .getip import lan_ips, best_ip from utils.bettersleep import better_sleep +from . import client """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -32,7 +33,7 @@ ANNOUNCE_LOOP_SLEEP = 30 -def learn_services(lan_client): +def learn_services(): """Take a list to infintely add lan service info to.""" sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) @@ -52,17 +53,17 @@ def learn_services(lan_client): if 'onionr' not in service_ips: continue service_ips = service_ips.replace('onionr-', '').split('-') + port = 0 for service in service_ips: try: ip_address(service) if not ip_address(service).is_private: raise ValueError if service in lan_ips: raise ValueError - if service in lan_client.peers: raise ValueError except ValueError: - service_ips.remove(service) - p = list(lan_client.peers) - lan_client.peers = list(set(service_ips + p)) + pass + else: + client.connect_peer(service) def advertise_service(specific_ips=None): @@ -71,10 +72,8 @@ def advertise_service(specific_ips=None): # for all packets sent, after three hops on the network the packet will not # be re-sent/broadcast (see https://www.tldp.org/HOWTO/Multicast-HOWTO-6.html) MULTICAST_TTL = 3 - if specific_ips is None: - ips = '-'.join(lan_ips) - else: - ips = '-'.join(specific_ips) + + ips = best_ip sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP) sock.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, MULTICAST_TTL) diff --git a/src/lan/server/__init__.py b/src/lan/server/__init__.py index fe45a386..29c4a931 100644 --- a/src/lan/server/__init__.py +++ b/src/lan/server/__init__.py @@ -2,19 +2,26 @@ LAN transport server thread """ +import ipaddress +import time +from threading import Thread + from gevent.pywsgi import WSGIServer from flask import Flask from flask import Response from flask import request +from flask import abort from onionrblocks.onionrblockapi import Block from httpapi.fdsafehandler import FDSafeHandler from netcontroller import get_open_port import config from coredb.blockmetadb import get_block_list -from lan.getip import best_ip +from lan.getip import best_ip, lan_ips from onionrutils import stringvalidators from httpapi.miscpublicapi.upload import accept_upload +import logger +from utils.bettersleep import better_sleep """ This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by @@ -29,7 +36,8 @@ from httpapi.miscpublicapi.upload import accept_upload You should have received a copy of the GNU General Public License along with this program. If not, see . """ - +ports = range(1337, 1340) +_start_time = time.time() class LANServer: def __init__(self, shared_state): @@ -41,6 +49,16 @@ class LANServer: self.host = best_ip self.port = None + @app.before_request + def dns_rebinding_prevention(): + if request.remote_addr in lan_ips or ipaddress.ip_address(request.remote_addr).is_loopback: + if time.time() - _start_time > 600: + abort(403) + if request.host != f'{self.host}:{self.port}': + logger.warn('Potential DNS rebinding attack on LAN server:') + logger.warn(f'Hostname {request.host} was used instead of {self.host}:{self.port}') + abort(403) + @app.route('/blist/