Merge next into Fred deprecate closer and do not delete class (used in plugins)

.github/workflows/package.yml

@@ -0,0 +1,68 @@
+name: Build debian package
+on:
+  push:
+    branches: [ 'master', 'next', 'debian-package' ]
+    tags:
+      - build**
+      - testing-**
+  pull_request:
+jobs:
+  build-deb:
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+        distribution: [temurin]
+        java: [17]
+
+    runs-on: '${{ matrix.os }}'
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+
+      - name: Wrapper validation
+        uses: gradle/actions/wrapper-validation@v4
+
+      - name: Speedup dpkg
+        run: sudo sh -c "echo 'force-unsafe-io' > /etc/dpkg/dpkg.cfg.d/force-unsafe-io"
+
+      - name: Fix 'Setup Java' action Gradle caching
+        run: |
+          sudo mkdir -p /tmp/fred-build/gradle
+          sudo ln -s /tmp/fred-build/gradle ~/.gradle
+          sudo chmod o+rwx ~/.gradle
+
+      - name: Setup Java
+        uses: actions/setup-java@v4
+        with:
+          cache: 'gradle'
+          distribution: '${{ matrix.distribution }}'
+          java-version: '${{ matrix.java }}'
+
+      - name: Install dependencies
+        run: |
+          sudo apt update
+          sudo apt install build-essential debhelper devscripts equivs -y
+
+      - name: Install package build dependencies
+        run: |
+          install_tool="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends --yes"
+          sudo mk-build-deps --install -r --tool="${install_tool}" debian/control
+
+      - name: Build package
+        run: |
+          sudo gbp buildpackage --git-ignore-branch --git-upstream-tag="HEAD" -us -uc --git-ignore-new || (EDITOR=true dpkg-source --commit . hack-1 && sudo gbp buildpackage --git-ignore-branch --git-upstream-tag="HEAD" -us -uc --git-ignore-new)
+
+      - name: Get package info
+        run: |
+          dpkg -I ../freenet*.deb
+          file ../freenet*.deb
+          cp ../freenet*.deb ./
+
+      - name: Provide Debian Package
+        uses: actions/upload-artifact@v4
+        with:
+          name: debian-package
+          path: freenet_*.deb
+          # Since .deb file is compressed archive itself,
+          # additional compression useless
+          compression-level: 0

.gitignore

@@ -15,8 +15,21 @@
 *.iml
 *.ipr
 *.iws
-.*
 *~
 \#*\#
 /override.properties
 /gradle.properties
+debhelper-build-stamp
+.debhelper
+*.deb
+*.dsc
+*.build
+*.buildinfo
+*.changes
+*.tar.gz
+*.log
+*.substvars
+/debian/freenet/*
+.gradle
+gradle
+debian/seednodes.fref

CONTRIBUTING.md

@@ -11,9 +11,9 @@ start with:
   List only bugs filed against this repository by selecting "Freenet" in the
   "Project" drop-down in the upper right. Do any of these look interesting?
 * Check the [projects](https://wiki.freenetproject.org/Projects) page.
-* Ask the [development mailing list](https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl)
-  or join us in [IRC](https://freenetproject.org/irc.html) - `#freenet` on
-  `chat.freenode.net`.
+* Ask the [development mailing list](https://freenetproject.org/pages/help.html#mailing-lists)
+  or join us in [IRC](https://web.libera.chat/?nick=Rabbit|?#freenet) - `#freenet` on
+  `irc.libera.chat`.
 
 # Code review
 
@@ -23,8 +23,7 @@ against the `next` branch. It's usually a good idea to work on a topic / feature
 branch that starts at `next` and is specific to the pull request. Once you have
 submitted a pull request, people will start reviewing it and providing feedback.
 If you don't hear anything for a while feel free to bring it to the attention of
-people in [IRC](https://freenetproject.org/irc.html) or the [mailing
-list](https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl) - we may
+people in [IRC](https://www.hyphanet.org/pages/help.html#chat-with-us) or the [mailing list](https://www.hyphanet.org/pages/help.html#mailing-lists) - we may
 have missed it.
 
 Code review helps improve code quality, ensures that multiple people know the
@@ -37,8 +36,8 @@ producing software that is readable, correct, and sufficiently efficient.
 Breaking API creates an immense amount of work for developers of other projects,
 so it is *not* something to be done lightly. If you feel you must make a change
 that breaks API and cannot maintain backwards compatibility, please first raise
-the issue with the community - the [mailing list](https://emu.freenetproject.org/cgi-bin/mailman/listinfo/devl)
-and [IRC](https://freenetproject.org/irc.html) are good places to contact us.
+the issue with the community - the [mailing list](https://www.hyphanet.org/pages/help.html#mailing-lists)
+and [IRC](https://www.hyphanet.org/pages/help.html#chat-with-us) are good places to contact us.
 
 Changing dependencies must be accompanied with changes in every
 installer and updates to `dependencies.properties` and existing

NEWS.md

@@ -1,6 +1,253 @@
 next:
 
-- 
+- Update Gradle to 8.9 thanks to Christophe!
+- replace length = 0 checks with isEmpty() — thanks to Juiceman!
+- do not recommend disabling js helpers in fproxy (these are reviewed)
+- add more resilient plugin list exception handling
+- Remove gc meddling that hasn’t been necessary for many Java releases
+- Add webp filter, thanks to Torusrxxx!
+- Update Japanese localization, thanks to qupo1!
+- Fix build info with gradle daemon, thanks to Bombe!
+- Improve IPv6 handling, thanks to Torusrxxx!
+- Hide no longer relevant new load management statistics, thanks to Torusrxxx!
+
+1498:
+
+# Short changelog:
+
+This release resolves the last blocker for Freenet / Hyphanet 0.8 by
+providing an official Debian package. Additionally it optimizes the
+networking and data transfer core and provides many improvements for
+website authors and user experience.
+
+
+Starting with this release, Freenet / Hyphanet has an official Debian
+package built automatically via github actions. This was the most
+important [high-impact-task][] and the last release blocker of version
+0.8 in our [Roadmap][]. Big thanks go to DC*!
+
+With this finally realized, the next step is to get in contact with
+the many privacy focussed distributions which build on Debian to make
+`hyphanet-fred` available where it is most important. Once this is
+done, tools which build on Hyphanet — like FMS, but also jSite and
+tools from pyFreenet — can be packaged to work out of the box, using
+Hyphanet as an ordinary background service. That’s a step towards
+Hyphanet as decentralized, privacy-preserving communication backend for
+other applications.
+
+Another step towards this is accepting the Schema hypha[net] to
+simplify writing browser extensions that forward hypha:-links to
+Hyphanet.
+
+
+The networking layer was optimized significantly. Searching packet
+types is often stopped early and common or cheaper checks are done
+before less common or time-consuming checks. This gives significant
+reductions of CPU load, especially for very fast nodes.
+
+Juiceman fixed a bug limiting MTU to 1280 where not needed.
+
+And recently failed and data not found cooldown times were reduced to
+5 minutes and 3 minutes, reducing one of the big annoyances when
+accessing a site quickly after upload.
+
+On the data transfer layer, healing was optimized. After 1495 strongly
+increased the amount of healing to keep large files available for
+longer, 1498 specializes healing to keys close to the node location.
+This reduces healing per file, but improves privacy, because healing
+inserts are then more similar to forwarding — they mostly send data
+close to the nodes location — and it reduces the network load of
+healing, because the specialized healing inserts need fewer hops to
+reach the optimal storage location in the network.
+
+
+In addition to these changes deep down, there are a number of directly
+visible improvements.
+
+The plugins KeepAlive and Sharesite are updated (the latter now uses
+the new Night Zen Garden style). The UPnP2 plugin is now visible in
+simple mode. It can replace UPnP and should work better. On the
+flipside the Library plugin is moved to advanced plugins, because it
+does not work reliably enough.
+
+The plugin list is easier to navigate by removing the defunct option
+to download plugins from the clearnet and by adding better styling.
+Downloading from the clearnet was an unnecessary privacy risk since
+we’ve been bundling essential plugins with the installer for a few
+years now.
+
+
+The noderef for friend-to-friend connections is shown in simple mode
+again, because it is robust enough with the changes in recent years.
+This should remove a barrier to adding direct connections and enabling
+fully confidential messages between friends.
+
+There are new configuration options to allow connecting via local
+services. That’s a step towards making it easy to add a second layer
+of security, for example confining connections to a local network.
+Thanks goes to s7r for these changes!
+
+When bandwidth detection fails, the upload bandwidth now defaults to
+160KiB/s. Also the NLM config is now disabled statically. This was a
+testing setup which could still be active in old nodes, but it would
+break connectivity nowadays.
+
+
+The default bookmarks include the Opennet SeedNodes statistics,
+the generate media site to create decentralized streaming sites, and
+the high-impact-tasks. The bookmarks are also re-ordered to be a
+better match for newcomers. Starting category: first steps, clean
+spider, Index of Indexes. For the software category ordered by ease of
+use from fproxy.
+
+For website authors, more CSS elements, selectors and combinators
+(`:checked`, `word-wrap: anywhere`, `focus-within`, `^=`, `$=`, `*=`,
+`>`, `+`, `~`) and additional HTML elements (`summary`, `details`,
+`<meta name="Viewport"...>`) are available. This strongly expands the
+possibilities of websites authors in Hyphanet, because Javascript or
+webassembly are no viable options in an environment where a privacy
+breach could put people at risk. We’ve seen with Java applets, that
+untrusted code will always break out of its containment. The CSS
+improvements in contrast provide a safe way to enable limited
+interactivity.
+
+Streaming support via m3u lists was improved to allow accessing
+segments of up to 200MiB.
+
+And using `-1` as version in a USK now properly finds version `0`, if
+this is the only existing version.
+
+
+There were a number of Java 21 fixes, including all our tests (thanks
+to Bombe!), and improvement to the github actions (thanks to
+AHOHNMYC).
+
+In addition to that there was a lot of polish. Bert Massop and
+Veniamin Fernandes replaced our homegrown CurrentTimeUTC with modern
+Java options. Alex fixed the pronoun used in strings. Bombe added
+getters for all direct field access in the node. Hiina reduced logging
+level of store warnings so no unneeded backtraces are created for node
+with large stores and Juiceman updated code to use more modern
+structures.
+
+Time-dependence of compressor selection was removed. This caused
+non-determinism for inserts and could cause keys to be
+non-reproducible on systems with faster or slower network.
+
+
+And finally the new [exe signing workflow][] we built to fulfill the
+requirements of SignPath, our new windows installer signing provider
+for the upcoming releases, runs the [verify-build script][] on every
+release to ensure that the jar we release has actually been built from
+the sources. This provides a second safety net, in addition to
+anonymous users running the script and posting the results (thanks to
+all who did this — please keep it up, otherwise people have to fully
+trust github). The release is not yet byte-by-byte reproducible,
+because the jar MANIFEST defines among other info the exact java
+version used to compile it, and the java version available differs by
+distribution and time, so it would get harder over time to verify the
+build.
+
+
+A special thanks goes to Bombe for many careful reviews!
+
+[high-impact-task]: https://github.com/hyphanet/wiki/wiki/High-Impact-tasks
+[Roadmap]: https://github.com/hyphanet/wiki/wiki/Roadmap
+[exe signing workflow]: https://github.com/hyphanet/sign-windows-installer
+[verify-build script]: https://github.com/hyphanet/scripts/blob/master/verify-build
+
+# Full changelog:
+
+
+## high impact tasks
+merge debian package as default build action thanks to DC*/desyncr! This resolves one of our high impact tasks.
+
+## Plugins
+Update KeepAlive to commit 86e47a101f26fd1d3be0437681a043aa4ae3f22c
+Update Sharesite to 0.5.1
+Move UPnP2 to normal plugins. It does not seem broken, but UPnP does
+Move Library plugin to advanced plugins because new users tend to get lost with it
+💄 Add better styling to the plugin list in winterfacey to make it easier to understand at a glance — thanks to Bombe
+🔥 Remove option to load plugins from central server — thanks to Bombe! This was an unnecessary privacy risk, since we’re already bundling essential plugins with the installer, and it made plugin handling harder to understand.
+
+## Bookmarks
+Add high-impact-tasks to bookmarks
+Add generate media site to the default bookmarks
+Add Opennet SeedNodes stats site
+Reorder starting bookmarks: FFS → clean spider → Index of Indexes
+Reorder default software bookmarks by ease of use from fproxy
+Disable activelink for Index of Indexes (workaround, because it fails)
+
+## Optimize networking and transfer layer
+break early when condition is met — thanks to Juiceman
+Check the HashCode before equals. This saves ~20% method-runtime.
+Re-order or’ed MessageFilters so the most likely is checked first
+specialize healing to keys close to the node
+fix healing decision: do not divide 0-1 by MAX_VALUE — thanks to Bombe for the review!
+Reduce recently failed and data not found wait times
+
+## Filters
+CSS: Fix: checked only the first char of the key part of CSS selectors, Add test that would catch too lax filtering.
+CSS: Support pseudo-element checked. This enables limited interactivity via CSS.
+CSS: Support the attribute selectors ^= $= *=, Add tests.
+CSS: Support Combinators > + and ~, add test for ~ and simplify the implementation
+CSS: Support word-wrap: anywhere and CSS selector focus-within.
+HTML: allow summary and details html element. Thanks to naejadu
+HTML: accept <meta name="Viewport" ...>, thanks to torusrxxx
+
+## Constants
+Show the noderef in basic-mode: it is now robust enough
+
+## Configuration
+accepting localhost in NodeIPPortDetector and allowBindToLocalhost configurable — thanks to s7r!
+Provide static methods for simpler boolean config creation
+Increase default bandwidth to 160KiB upload, when detection fails
+disable setting for new-load-management (NLM broke nodes)
+add utility to disable a config option, thanks to Bombe
+
+## Misc
+add m3u-player insertion test: is added at end of body
+[CI] Update actions, fix actions cache
+♻️ add and use getters and setters for access to node fields
+Increase max transparent passthrough to 200MiB links in m3u-lists.
+Remove time-dependence of compressor selection. This caused
+non-determinism for inserts and could cause keys to be
+non-reproducible.
+improve date object construction in CurrentTimeUTC.get()
+Support Schema hypha[net] to simplify writing browser extensions that forward hypha://-links to Hyphanet.
+polish: show datastore size warning with GiB suffix
+Remove hash generation to native big integer to reduce dependencies. This had come in when merging an old pull request and added a new dependency without need.
+
+## Polish
+Replace indexOf with .contains()
+Change more string comparisons into .isEmpty() checks
+Capitalize L in literal longs
+Add missing Global flag to GetFailed FCP message. Thanks to oishii and Bombe!
+Fix links to mailing lists and IRC in CONTRIBUTING.md — thanks to Juiceman!
+Deprecated custom datetime handling class CurrentTimeUTC
+replace GregorianCalendar with java.time.OffsetDateTime in CurrentTimeUTC — thanks to Veniamin Fernandes
+Refine ClientRequestSelectorTest, PersistentJobRunnerImplTest, SplitFileFetcherStorageTest, and SplitFileInserterStorageTest — thanks to Veniamin Fernandes
+Change deprecated jcenter() maven repo to mavenCentral() — thanks to Veniamin Fernandes
+Added tests for PebbleUtils — thanks to Bombe!
+🐛 Fix NPE when subsets are not initialized — thanks to Bombe
+reduce logging for too many excluded sub-arrays thanks to Hiina
+fix the flag size of nepal — thanks to Percept0r@NYZkOs7eQ…!
+Switch swiss flag to civil and state ensign — thanks to Percept0r@NY
+
+## Fixes
+Thanks to Bombe all our tests work again on Java 21!
+🐛 Fix JarClassLoader’s ability to work with ServiceLoader — thanks to Bombe!
+gzip: replace test workaround by fixing the output of the compressor — thanks to Bombe for the SingleOffsetReplacingOutputStream!
+fix: invalid max store size showed bytes with GiB suffix
+
+
+Also a special thanks to Bombe for many careful reviews!
+
+
+1497:
+
+- fixed severe path folding vulnerability.
+
 
 1496: