Skip to content

Conversation

@peter-lawrey
Copy link
Member

@peter-lawrey peter-lawrey commented Nov 19, 2025

Functional changes

  1. Dependency & configuration surface
  • Bump net.openhft:third-party-bom from 3.27ea53.27ea7.

    • Location: root pom.xml.
    • Effect: updates a number of transitive third-party dependencies via the BOM. Behavioural impact comes from those dependency upgrades; downstreams should verify their dependency trees if they rely on exact versions.
  1. CLI and dump output encoding
  • Standardise CLI help output on UTF-8 for Chronicle tools:

    • ChronicleHistoryReaderMain#printHelpAndExit(...)
    • ChronicleReaderMain#printHelpAndExit(...)
    • internal.writer.ChronicleWriterMain#printHelpAndExit(...)
  • Standardise dump output on UTF-8 when writing to a file:

    • InternalDumpMain.dump(String) now creates PrintStream(FILE, "UTF-8"), with a guarded fallback if the charset is unavailable.
  • Behavioural effect: CLI and dump output will now consistently use UTF-8 when redirected, avoiding platform-default encoding surprises.

  1. Queue directory handling and metadata paths
  • More robust directory creation and warnings:

    • SingleChronicleQueue and SingleChronicleQueueBuilder:

      • If path does not exist and mkdirs() fails (and the directory still does not exist), emit a Jvm.warn() instead of failing silently.

      • SingleChronicleQueueBuilder.builder(File, WireType):

        • Special-cases file.toString().equals("") to mean “current directory”, logs a warning, and normalises to new File(".").
      • metapath():

        • Uses the same “mkdirs + warn on failure” pattern when creating metadata.cq4t.
  • Behavioural effect: success paths are unchanged; misconfigured or unwritable directories now produce explicit warnings instead of silent failure.

  1. Index-check configuration hook
  • Introduce a managed index-check flag in QueueSystemProperties:

    • New methods:

      • public static boolean checkIndex() – reads a cached queue.check.index value.
      • public static void setCheckIndex(boolean enabled)test-only, allows overriding the flag.
    • CHECK_INDEX is now final and deprecated, backed by checkIndex().

    • All internal assertions have been switched from QueueSystemProperties.CHECK_INDEX to QueueSystemProperties.checkIndex().

  • Behavioural effect: runtime semantics remain governed by queue.check.index, but tests can toggle the behaviour safely without mutating a writable static.

  1. Internal lock and tailer semantics
  • TableStoreWriteLock lock acquisition refactor:

    • Extracted CAS retry logic into retryLockAcquisition(), with pauser reset in a finally block.
    • Semantics are unchanged: keep retrying until success or TimeoutException, but pauser lifecycle is clearer and less error-prone.
  • StoreTailer robustness improvements:

    • incrementIndex() now has an explicit default branch that throws IllegalStateException for unsupported TailerDirection values.
    • close() now uses isData() rather than isPresent() && !isMetaData() when deciding to advance the index.
  • StoreAppender EOF and index checks:

    • cycleHasEOF() uses a local wireLocal and guards the Bytes reservation/release around endOfWire(...).
    • All index-check asserts now call QueueSystemProperties.checkIndex().
  1. Deprecated, test-only and internal API markers

(Behaviour unchanged; these are purely API signalling changes.)

  • Marked as deprecated, test-only or slated for removal in 2027:

    • ChronicleQueue:

      • clear(), dump(OutputStream, long, long), lastIndexMSynced() and lastIndexMSynced(long).
    • ExcerptTailer.readAfterReplicaAcknowledged().

    • RollingChronicleQueue: storeForCycle(...), lastCycle(), nextCycle(...), countExcerpts(...).

    • WireStore: multiple low-level methods (writePosition(long), moveToIndexForRead(...), moveToStartForRead(...), sequenceForPosition(...), setPositionForSequenceNumber(...), writeEOF(...), linearScanTo(...), moveToEndForRead(...), initIndex(...), dumpHeader(), dataVersion()).

    • Indexing’s inspection methods (index counts, spacing, linear scan counters, etc.).

    • Various builders and helpers (SingleChronicleQueueBuilder.rollTime(LocalTime), queueOffsetSpec(...), ring-buffer tuning hooks, forceDirectoryListingRefreshIntervalMs(...), appenderListener(...), SingleChronicleQueue.chunkCount(), storeFileListener(), storeSupplier(), acquireAppender(), tableStoreGet(...), SingleChronicleQueueStore.dump(...)).

    • Reader/HistoryReader/Reader factory and fluent helper methods now explicitly tagged as test-only.

  • Behavioural effect: none at runtime; callers now have clear guidance on which APIs are legacy or test-focused and scheduled for removal.


Non-functional changes

  1. Documentation and AsciiDoc updates
  • README / docs structure alignment:

    • Image path updated to src/main/docs/images/Queue_line.png.
    • GitHub links for Big Data, FAQ, How it works, and Utilities updated to point into src/main/docs/.
    • Fix link to system properties (docs/system-properties.adoc).
  • New testing strategy doc:

    • Added src/main/docs/testing-strategy.adoc:

      • Describes unit / integration / performance test layers.
      • Links tests back to project-requirements.adoc (QUEUE-FN-*, QUEUE-NF-*) and decision-log.adoc.
      • Provides guidance on resource management, leak detection and test design.
  • System properties and codegen diagnostics:

    • docs/system-properties.adoc:

      • Adds entries for:

        • wire.generator.v2 – enable v2 method-writer code generator.
        • disableProxyCodegen – disable proxy fallback (fail fast when codegen breaks).
    • docs/antora/modules/demos/pages/message-history.adoc:

      • New section “Debugging method-writer fallback warnings” explaining how to reproduce and triage Failed to compile generated method writer – falling back to proxy method writer.
  • General docs polish:

    • Various grammar and spacing fixes (glossary.adoc, quick-start, roll-cycle and tailing docs).
    • DumpMain Antora anchor fixed to #dumpmain.
  1. Improved diagnostics for method-writer failures
  • Test harness logging in QueueTestCommon:

    • Tracks recorded exceptions per test via Jvm.recordExceptions(false).

    • On test teardown, if a recorded exception message contains "Failed to compile generated method writer":

      • Logs a warning with:

        • Test name,
        • java.version,
        • wire.generator.v2 and disableProxyCodegen values,
        • Exception message and throwable.
  • Benefit: makes intermittent codegen fallbacks on Java 8 significantly easier to diagnose without trawling full logs.

  1. Internal quality, readability and test improvements
  • Encoding / charset correctness:

    • Use StandardCharsets.UTF_8 and Files.newInputStream(...) where we inspect /proc/self/maps.
    • Replace new String(byte[]) with ByteArrayOutputStream.toString() in tests.
  • Busy-wait reductions:

    • Replace several while (...) ; loops in tests/benchmarks with Jvm.nanoPause() or Thread.yield() for more cooperative waiting.
  • Refactoring & duplication removal in tests:

    • Extracted helpers for:

      • Tailer navigation assertions in TestDeleteQueueFile.
      • Binary search test execution and data setup in ChronicleReaderTest.
      • “Half-written message” tailer assertions in CheckHalfWrittenMsgNotSeenByTailerTest.
      • Partial update queue creation in PartialUpdateTest.
      • Two-message roll tests in SingleChronicleQueueTest.
  • Static analysis & style clean-ups:

    • Simplify boolean assertions (use assertFalse instead of negated assertTrue).
    • Prefer assertSame where type identity is important.
    • Move magic constants into final fields (e.g. histogram capacities, buffer sizes).
    • Remove unused local variables and dead code.
    • Fix logging format strings to use parameterised SLF4J style in tests and perf harnesses.
  • Tests now express intent more clearly:

    • Use Assert.assertThrows(...) rather than manual try/catch fail() patterns.
    • Clarify JUnit expectations in Windows-specific tests (FileUtilTest.stateWindows).
    • Consistent use of final on temp folders, histograms, and other test fixtures.

Overall, this PR does not change core queue behaviour but:

  • Clarifies and strengthens documentation around testing and method-writer diagnostics.
  • Normalises CLI and dump encodings to UTF-8.
  • Makes directory creation failures and index-check configuration more explicit.
  • Marks a broad set of internal/test-only APIs as deprecated ahead of a 2027 removal window.
  • Tidies many tests and internal tools for better readability and reduced flakiness.

@peter-lawrey peter-lawrey changed the title Adv/develop Add architecture docs and small robustness fixes for Chronicle Queue Nov 19, 2025
@peter-lawrey peter-lawrey changed the title Add architecture docs and small robustness fixes for Chronicle Queue Document Chronicle Queue architecture, add AI agent guides, and harden CLI / queue error handling Nov 20, 2025
@peter-lawrey peter-lawrey changed the title Document Chronicle Queue architecture, add AI agent guides, and harden CLI / queue error handling Architecture docs and small robustness fixes for Chronicle Queue Nov 24, 2025
… lock acquisition and ensure proper pauser reset
@sonarqubecloud
Copy link

sonarqubecloud bot commented Dec 1, 2025

# Conflicts:
#	README.adoc
#	docs/DISCLAIMER.adoc
#	docs/FAQ.adoc
#	docs/antora/modules/command-line/pages/command_line_tools.adoc
#	docs/antora/modules/demos/pages/demos.adoc
#	docs/utilities.adoc
#	src/main/docs/decision-log.adoc
#	src/main/docs/project-requirements.adoc
#	src/main/docs/security-review.adoc
#	src/main/java/net/openhft/chronicle/queue/impl/ExcerptContext.java
#	src/main/java/net/openhft/chronicle/queue/impl/single/namedtailer/IndexUpdaterFactory.java
#	src/main/java/net/openhft/chronicle/queue/internal/main/InternalBenchmarkMain.java
#	src/test/java/net/openhft/chronicle/queue/ChronicleQueueLatencyDistribution.java
#	src/test/java/net/openhft/chronicle/queue/ChronicleRollingIssueTest.java
#	src/test/java/net/openhft/chronicle/queue/CreateAtIndexTest.java
#	src/test/java/net/openhft/chronicle/queue/impl/single/stress/RollCycleMultiThreadStressDoubleBufferTest.java
#	src/test/java/net/openhft/chronicle/queue/impl/single/stress/RollCycleMultiThreadStressSharedWriterQueueTest.java
#	src/test/java/net/openhft/chronicle/queue/internal/reader/ChronicleReaderTest.java
@peter-lawrey peter-lawrey changed the base branch from develop to adv/javadoc December 11, 2025 14:50
@peter-lawrey peter-lawrey changed the title Architecture docs and small robustness fixes for Chronicle Queue Clarify testing strategy, improve docs and CLI encoding, and add deprecation markers for test-only APIs Dec 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants