Skip to content

Proposal: NBT APIs #278

@wu-vincent

Description

@wu-vincent

Proposal: NBT APIs

Status: Draft (Proposed)
Updated: 2025-11-11
Revisions included: explicit-typed • no-implicit-empty-compound • concise CompoundTag init • kind-reassignment-allowed • no runtime range checks • templated scalar wrapper • std::monostate End • full Tag declaration • getters return wrapper refs (value = const, containers = mutable) • ListTag/CompoundTag classes with private members • Python auto-caster (value tags read-only; containers mutable) • value tags immutable in place • typed arrays are mutable containers via ArrayTag<T> with braced initialisersequality operators (==, !=) for all tagsis_mutable_tag_v type trait for containers

Scope: Minecraft Bedrock NBT (LE-NBT). Text I/O is SNBT with JSON-style knobs.


1 Abstract

A strongly typed, JSON-like DOM for Bedrock NBT in C++ and Python, with explicit typing only, no implicit empty compounds, and ergonomic wrappers.

  • C++ Tag — a dynamic node backed by std::variant of explicit wrapper types.
  • Scalars (value tags) are immutable in place; to change, replace the node at its parent key.
  • Containers are mutable in place: ListTag, CompoundTag, and typed arrays via ArrayTag<T>.
  • Typed arrays support clean braced initialisers, e.g. ByteArrayTag{1,2,3}.
  • Wrappers (ValueTag<T> for scalars) implicitly convert to payload types (read-only).
  • Lists are homogeneous with retro-fill; compounds are deterministic; kind reassignment allowed at compound keys.
  • No runtime numeric range checks.
  • End is represented by std::monostate.
  • Python API auto-caster: tag["k"] returns wrapper/proxy; value-tag proxies read-only, container proxies mutable.
  • Equality: deep, structural equality across all tags (==, !=).
  • Trait: is_mutable_tag_v<T> encodes which wrappers are mutable (containers).

2 Types & Tag Kinds

Supported kinds:

End, Byte, Short, Int, Long, Float, Double, ByteArray, String, List, Compound, IntArray

2.1 TagType enum

namespace endstone::nbt {

enum class TagType : std::uint8_t {
    End       = 0,
    Byte      = 1,
    Short     = 2,
    Int       = 3,
    Long      = 4,
    Float     = 5,
    Double    = 6,
    ByteArray = 7,
    String    = 8,
    List      = 9,
    Compound  = 10,
    IntArray  = 11,
};

3 Scalar Wrappers (Value Tags)

3.1 ValueTag<T> — read-only in place with equality

template <typename T>
class ValueTag {
public:
    using value_type = T;

    constexpr ValueTag() = default;
    constexpr explicit ValueTag(const T& v) : value_(v) {}
    constexpr explicit ValueTag(T&& v) : value_(std::move(v)) {}

    // Read-only implicit conversion to payload
    operator const T&() const noexcept { return value_; }
    const T& value() const noexcept { return value_; }

    // Equality with same wrapper
    friend constexpr bool operator==(const ValueTag& a, const ValueTag& b) noexcept {
        return a.value_ == b.value_;
    }
    friend constexpr bool operator!=(const ValueTag& a, const ValueTag& b) noexcept {
        return !(a == b);
    }
    // Equality with underlying payload
    friend constexpr bool operator==(const ValueTag& a, const T& b) noexcept {
        return a.value_ == b;
    }
    friend constexpr bool operator==(const T& a, const ValueTag& b) noexcept {
        return a == b.value_;
    }
    friend constexpr bool operator!=(const ValueTag& a, const T& b) noexcept {
        return !(a == b);
    }
    friend constexpr bool operator!=(const T& a, const ValueTag& b) noexcept {
        return !(a == b);
    }

private:
    T value_;
};

Concrete scalar wrappers:

using EndTag    = std::monostate;               // sentinel, not a payload
using ByteTag   = ValueTag<std::uint8_t>;
using ShortTag  = ValueTag<std::int16_t>;
using IntTag    = ValueTag<std::int32_t>;
using LongTag   = ValueTag<std::int64_t>;
using FloatTag  = ValueTag<float>;
using DoubleTag = ValueTag<double>;
using StringTag = ValueTag<std::string>;

Mutation rule: ValueTag payloads cannot be mutated in place. Update via parent assignment:

obj["name"] = StringTag{"NewName"}; // replace node

4 Containers (Mutable In-Place)

4.1 ArrayTag<T> — templated typed arrays with braced initialisers & equality

template <typename Elem>
class ArrayTag {
public:
  using value_type     = Elem;
  using size_type      = std::size_t;
  using storage_type   = std::vector<value_type>;
  using iterator       = typename storage_type::iterator;
  using const_iterator = typename storage_type::const_iterator;

  // Constructors
  ArrayTag() = default;
  explicit ArrayTag(storage_type v) : v_(std::move(v)) {}
  template<class It> ArrayTag(It first, It last) : v_(first, last) {}
  ArrayTag(std::initializer_list<value_type> init) : v_(init) {} // {1,2,3}

  // Capacity
  bool      empty() const noexcept { return v_.empty(); }
  size_type size()  const noexcept { return v_.size(); }
  void      clear() noexcept { v_.clear(); }

  // Element access
  value_type&       at(size_type i)             { if (i >= v_.size()) throw std::out_of_range("ArrayTag::at"); return v_[i]; }
  const value_type& at(size_type i) const       { if (i >= v_.size()) throw std::out_of_range("ArrayTag::at"); return v_[i]; }
  value_type&       operator[](size_type i)     { return v_[i]; }
  const value_type& operator[](size_type i) const { return v_[i]; }
  value_type*       data() noexcept             { return v_.data(); }
  const value_type* data() const noexcept       { return v_.data(); }

  // Modifiers
  void push_back(value_type v)                  { v_.push_back(v); }
  void resize(size_type n)                      { v_.resize(n); }
  void resize(size_type n, value_type fill)     { v_.resize(n, fill); }
  template<class It> void assign(It first, It last) { v_.assign(first, last); }
  iterator insert(const_iterator pos, value_type v) { return v_.insert(pos, v); }
  iterator erase(const_iterator pos)            { return v_.erase(pos); }

  // Iteration
  iterator begin() noexcept { return v_.begin(); }
  iterator end()   noexcept { return v_.end(); }
  const_iterator begin() const noexcept { return v_.begin(); }
  const_iterator end()   const noexcept { return v_.end(); }
  const_iterator cbegin() const noexcept { return v_.cbegin(); }
  const_iterator cend()   const noexcept { return v_.cend(); }

  // Equality
  friend bool operator==(const ArrayTag& a, const ArrayTag& b) noexcept { return a.v_ == b.v_; }
  friend bool operator!=(const ArrayTag& a, const ArrayTag& b) noexcept { return !(a == b); }
  friend bool operator==(const ArrayTag& a, const storage_type& b) noexcept { return a.v_ == b; }
  friend bool operator==(const storage_type& a, const ArrayTag& b) noexcept { return a == b.v_; }
  friend bool operator!=(const ArrayTag& a, const storage_type& b) noexcept { return !(a == b); }
  friend bool operator!=(const storage_type& a, const ArrayTag& b) noexcept { return !(a == b); }

private:
  storage_type v_;
};

// Typed-array aliases
using ByteArrayTag = ArrayTag<std::uint8_t>;
using IntArrayTag  = ArrayTag<std::int32_t>;

4.2 ListTag — homogeneous, retro-fill, private members, equality

class Tag; // fwd

class ListTag {
public:
  using size_type       = std::size_t;
  using container_type  = std::vector<Tag>;
  using iterator        = container_type::iterator;
  using const_iterator  = container_type::const_iterator;

  // Constructors
  ListTag();
  ListTag(std::initializer_list<Tag> init);
  explicit ListTag(size_type n);
  ListTag(size_type n, const Tag& fill);
  template<class It> ListTag(It first, It last);

  // Capacity
  bool      empty() const noexcept;
  size_type size()  const noexcept;

  // Element typing
  TagType   element_type() const noexcept;     // TagType::End until fixed
  bool      element_type_fixed() const noexcept;
  void      fix_element_type(TagType t);       // explicit fix; retro-fills End placeholders

  // Element access
  Tag&       at(size_type i);
  const Tag& at(size_type i) const;
  Tag&       operator[](size_type i);
  const Tag& operator[](size_type i) const;

  // Modifiers
  void       clear() noexcept;
  void       push_back(const Tag& v);
  void       push_back(Tag&& v);
  template<class... Args> Tag& emplace_back(Args&&... args);
  iterator   insert(const_iterator pos, const Tag& v);
  iterator   insert(const_iterator pos, Tag&& v);
  iterator   erase(const_iterator pos);
  iterator   erase(const_iterator first, const_iterator last);
  void       resize(size_type n);
  void       resize(size_type n, const Tag& fill);
  template<class It> void assign(It first, It last);

  // Iteration
  iterator        begin() noexcept;
  iterator        end()   noexcept;
  const_iterator  begin() const noexcept;
  const_iterator  end()   const noexcept;
  const_iterator  cbegin() const noexcept;
  const_iterator  cend()   const noexcept;

  // Equality
  friend bool operator==(const ListTag& a, const ListTag& b) noexcept { return a.elements_ == b.elements_; }
  friend bool operator!=(const ListTag& a, const ListTag& b) noexcept { return !(a == b); }

private:
  TagType        element_type_{ TagType::End };
  container_type elements_;
};

4.3 CompoundTag — deterministic order, private members, equality

class CompoundTag {
public:
  using map_type        = std::map<std::string, Tag, std::less<>>;
  using size_type       = std::size_t;
  using iterator        = map_type::iterator;
  using const_iterator  = map_type::const_iterator;

  // Constructors
  CompoundTag();
  CompoundTag(std::initializer_list<std::pair<std::string, Tag>> init);
  template<class It> CompoundTag(It first, It last);

  // Capacity
  bool      empty() const noexcept;
  size_type size()  const noexcept;

  // Lookup / element access
  Tag&        at(std::string_view key);
  const Tag&  at(std::string_view key) const;
  Tag&        operator[](const std::string& key); // inserts End if absent
  Tag&        operator[](std::string&& key);
  bool        contains(std::string_view key) const noexcept;
  size_type   count(std::string_view key) const noexcept;

  // Modifiers
  void        clear() noexcept;
  std::pair<iterator, bool> insert(const std::pair<const std::string, Tag>& v);
  std::pair<iterator, bool> insert(std::pair<const std::string, Tag>&& v);
  template<class P> std::pair<iterator, bool> insert(P&& v);
  template<class... Args> std::pair<iterator, bool> try_emplace(std::string key, Args&&... ctor_args);
  template<class M> std::pair<iterator, bool> insert_or_assign(std::string key, M&& obj);
  iterator    erase(const_iterator pos);
  size_type   erase(std::string_view key);
  iterator    erase(const_iterator first, const_iterator last);
  void        swap(CompoundTag& other) noexcept;
  void        merge(CompoundTag& source);
  void        merge(CompoundTag&& source);

  // Iteration
  iterator        begin() noexcept;
  iterator        end()   noexcept;
  const_iterator  begin() const noexcept;
  const_iterator  end()   const noexcept;
  const_iterator  cbegin() const noexcept;
  const_iterator  cend()   const noexcept;

  // Equality (deep structural)
  friend bool operator==(const CompoundTag& a, const CompoundTag& b) noexcept { return a.entries_ == b.entries_; }
  friend bool operator!=(const CompoundTag& a, const CompoundTag& b) noexcept { return !(a == b); }

private:
  map_type entries_;
};

5 Type Traits

5.1 is_mutable_tag_v

Encodes which tag wrapper types are mutable in place (i.e., containers). Defaults to false.

template <typename T> struct is_mutable_tag : std::false_type {};
template <> struct is_mutable_tag<ListTag>      : std::true_type {};
template <> struct is_mutable_tag<CompoundTag>  : std::true_type {};
template <> struct is_mutable_tag<ByteArrayTag> : std::true_type {};
template <> struct is_mutable_tag<IntArrayTag>  : std::true_type {};

template <typename T>
inline constexpr bool is_mutable_tag_v = is_mutable_tag<T>::value;

This trait is used to drive reference return types in Tag::get()/get_if() and to centralise mutability rules.


6 Dynamic Node (Tag)

class Tag {
public:
  using Storage = std::variant<
    std::monostate,                // End
    ByteTag, ShortTag, IntTag, LongTag,
    FloatTag, DoubleTag,
    StringTag,
    ByteArrayTag, IntArrayTag,     // containers
    ListTag, CompoundTag           // containers
  >;

  // Constructors
  Tag() noexcept : storage_(std::monostate{}) {}
  explicit Tag(ByteTag v)      : storage_(std::move(v)) {}
  explicit Tag(ShortTag v)     : storage_(std::move(v)) {}
  explicit Tag(IntTag v)       : storage_(std::move(v)) {}
  explicit Tag(LongTag v)      : storage_(std::move(v)) {}
  explicit Tag(FloatTag v)     : storage_(std::move(v)) {}
  explicit Tag(DoubleTag v)    : storage_(std::move(v)) {}
  explicit Tag(StringTag v)    : storage_(std::move(v)) {}
  explicit Tag(ByteArrayTag v) : storage_(std::move(v)) {}
  explicit Tag(IntArrayTag v)  : storage_(std::move(v)) {}
  explicit Tag(ListTag v)      : storage_(std::move(v)) {}
  explicit Tag(CompoundTag v)  : storage_(std::move(v)) {}

  // Kind / size
  TagType     type()  const noexcept;     // map variant index to TagType
  std::size_t size()  const noexcept;     // 0 for non-containers; container size otherwise
  bool        empty() const noexcept;     // true for End or empty containers

  // Self-shaping access
  Tag&        operator[](std::string_view key);     // End -> Compound
  Tag&        operator[](std::size_t index);        // End -> List
  Tag&        at(std::string_view key);             // throws if not compound/missing
  const Tag&  at(std::string_view key) const;
  Tag&        at(std::size_t index);                // throws if not list/OOB
  const Tag&  at(std::size_t index) const;
  bool        contains(std::string_view key) const noexcept; // only if compound

  // Getters (wrappers) — uses is_mutable_tag_v
  // - Value wrappers: return **const T&** (read-only)
  // - Containers (is_mutable_tag_v<T> == true): **T&** (mutable)
  template <typename T>
  decltype(auto) get() {
      if (auto p = std::get_if<T>(&storage_)) {
          if constexpr (is_mutable_tag_v<T>) {
              return *p;                 // T&
          } else {
              return static_cast<const T&>(*p); // const T&
          }
      }
      throw std::runtime_error("Tag::get<T>() kind mismatch");
  }
  template <typename T>
  decltype(auto) get() const {
      if (auto p = std::get_if<T>(&storage_)) {
          return static_cast<const T&>(*p); // const for all in const context
      }
      throw std::runtime_error("Tag::get<T>() kind mismatch");
  }

  // get_if mirrors constness above, trait-driven
  template <typename T>
  auto get_if() noexcept {
      if (auto p = std::get_if<T>(&storage_)) {
          if constexpr (is_mutable_tag_v<T>) return p;                  // T*
          else return const_cast<const T*>(p);                          // const T*
      }
      return static_cast<std::conditional_t<is_mutable_tag_v<T>, T*, const T*>>(nullptr);
  }
  template <typename T>
  const T* get_if() const noexcept { return std::get_if<T>(&storage_); }

  // Equality (deep, via variant)
  friend bool operator==(const Tag& a, const Tag& b) noexcept { return a.storage_ == b.storage_; }
  friend bool operator!=(const Tag& a, const Tag& b) noexcept { return !(a == b); }

  // Serialization
  struct DumpOptions {
    int   indent = -1; char indent_char = ' ';
    bool  ensure_ascii = false;
    bool  space_after_colon = true;
    bool  space_after_comma = true;
    bool  sort_keys = false;
  };
  std::string dump(const DumpOptions& = {}) const;

  struct ParseOptions {
    bool allow_trailing_commas = false;
    bool allow_comments = false;
    bool allow_nan_inf = false;
    bool strict_lists_homogeneous = true;
  };
  static Tag parse(std::string_view, const ParseOptions& = {});

  struct ReadOptions  { bool littleEndian=true; bool expectNamedRoot=true; bool compressed=false; };
  struct WriteOptions { bool littleEndian=true; bool namedRoot=true;        bool compressed=false; int zlibLevel=6; };
  static Tag  read(std::span<const std::byte>, const ReadOptions& = {});
  static void write(std::vector<std::byte>& out, const Tag& root, const WriteOptions& = {});

private:
  Storage storage_;
};

7 Assignment Semantics (C++ explicit only)

  • obj["Damage"] = ShortTag{6};

  • obj["Damage"] = 6; (no guessing)

  • Kind reassignment allowed for compounds (replaces node).

  • Lists remain homogeneous after first non-End element fixes element_type.

  • Value tags are read-only in place — to change, replace:

    obj["name"] = StringTag{"Conduit"};
  • Containers are mutable in place (is_mutable_tag_v<T> == true, including typed arrays):

    auto& ba = obj["bytes"].get<ByteArrayTag>();
    ba.push_back(255);
    ba[3] = 17;
  • No implicit empty compounds:

    obj["Test"] = CompoundTag{}; // ok
    obj["Test"] = {};            // not allowed

8 SNBT & Binary I/O (C++)

std::string Tag::dump(const DumpOptions&) const;
Tag         Tag::parse(std::string_view, const ParseOptions&);
Tag         Tag::read(std::span<const std::byte>, const ReadOptions&);
void        Tag::write(std::vector<std::byte>& out, const Tag& root, const WriteOptions&);
  • SNBT supports numeric suffixes (b,s,L,f,d) and array forms ([B; …], [I; …]).
  • {} is valid text for an empty compound; programmatic {} is not accepted—use CompoundTag{}.
  • No runtime numeric range checks.

9 Python API (endstone.nbt)

9.1 Core classes

End (sentinel), ByteTag, ShortTag, IntTag, LongTag, FloatTag, DoubleTag, StringTag (read-only in place), ByteArrayTag (mutable), IntArrayTag (mutable), ListTag (mutable), CompoundTag (mutable), Tag.

9.2 Explicit construction & assignment

  • No primitive guessing (use wrappers).
  • No implicit empty compounds (use nbt.CompoundTag()).

9.3 Automatic caster on indexing

  • __getitem__ returns a wrapper/proxy bound to the underlying node.
  • Value-tag proxies are read-only (.value is read-only; assign a new wrapper to change).
  • Container proxies are mutable (lists, compounds, byte/int arrays).

Examples:

from endstone import nbt

cfg = nbt.CompoundTag({
    "name":  nbt.StringTag("Beacon"),
    "level": nbt.IntTag(4),
    "bytes": nbt.ByteArrayTag([1,2,3]),
})
cfg["effects"] = nbt.ListTag()

# Read (value tags)
s = cfg["name"].value
n = cfg["level"].value

# Replace (not mutate in place)
cfg["name"]  = nbt.StringTag("Conduit")
cfg["level"] = nbt.IntTag(5)

# Containers are mutable
ba = cfg["bytes"]             # ByteArrayTag proxy
ba.append(255)
ba[1] = 7

effects = cfg["effects"]
effects.append(nbt.StringTag("Speed"))

9.4 I/O

def dumps(value, *, ...) -> str: ...
def loads(text: str, *, ...) -> nbt.Tag: ...
def read(buf: bytes, *, little_endian=True, expect_named_root=True, compressed=False) -> nbt.Tag: ...
def write(value, *, little_endian=True, named_root=True, compressed=False, zlib_level=6) -> bytes: ...

10 Semantics & Invariants

  • Self-shaping: EndCompound on first key write; EndList on first index write.
  • Explicit typing: first non-End list element fixes element_type; earlier End placeholders retro-filled.
  • Deterministic compounds: ordered map.
  • Serialization invariant: non-empty lists contain no End.
  • Mutability rule: value tags read-only; containers (is_mutable_tag_v<T>) mutable.
  • Equality: deep structural equality across all tags; cross-kind comparisons are false.

11 Examples

11.1 Concise compound and typed-array braced init (C++)

using namespace endstone::nbt;

Tag cfg = CompoundTag{
  { "name",  StringTag{"Beacon"} },
  { "level", IntTag{4} },
  { "bytes", ByteArrayTag{1,2,3} },            // <- braced init
};

// Read value tags (implicit conversion from const wrapper)
std::string name = cfg["name"].get<StringTag>();
int level = cfg["level"].get<IntTag>();

// Replace value tags
cfg["name"]  = StringTag{"Conduit"};
cfg["level"] = IntTag{5};

// Mutate typed arrays in place
auto& ba = cfg["bytes"].get<ByteArrayTag>();
ba.push_back(255);
ba[1] = 7;

11.2 Nested compound

Tag root = CompoundTag{ 
  { "display", CompoundTag{
      { "Name",  StringTag{"Sword"} },
      { "Lore",  ListTag{ StringTag{"Sharp"}, StringTag{"Ancient"} } }
  }},
  { "attributes", CompoundTag{} },
  { "entities",   ListTag{} },
  { "matrix",     ListTag{
      ListTag{ IntTag{1}, IntTag{2}, IntTag{3} },
      ListTag{ IntTag{4}, IntTag{5}, IntTag{6} }
  }}
};

11.3 Equality quick checks

// ValueTag
static_assert(ByteTag{5} == ByteTag{5});
static_assert(ByteTag{5} != ByteTag{6});
static_assert(StringTag{"hi"} == std::string("hi"));

// ArrayTag
ByteArrayTag a{1,2,3}, b{1,2,3}, c{3,2,1};
assert(a == b && a != c);

// ListTag / CompoundTag structural equality
assert( ListTag{ IntTag{1}, IntTag{2} } == ListTag{ IntTag{1}, IntTag{2} } );

assert( CompoundTag{
  { "Damage", ShortTag{5} },
  { "Name", StringTag{"Sword"} }
} == CompoundTag{
  { "Damage", ShortTag{5} },
  { "Name", StringTag{"Sword"} }
} );

// Tag (variant) deep equality
assert( Tag(CompoundTag{ {"x", IntTag{10}} }) == Tag(CompoundTag{ {"x", IntTag{10}} }) );

12 Errors & Diagnostics

  • Wrong container access / OOB → std::out_of_range (C++), IndexError/KeyError (Py).
  • Getter kind mismatch (C++) → std::runtime_error.
  • Attempting in-place mutation of a value tag in Python → AttributeError / TypeError.
  • Implicit primitives or {} assignment rejected (C++: std::runtime_error; Py: ValueError).
  • Unsupported LongArraystd::runtime_error / NotImplementedError.
  • Parse errors are fatal.

13 Security

  • Defensive limits on parse depth and compressed size; fail-closed decompression.
  • Reject untyped or malformed inputs early with clear diagnostics.

14 Must-pass Behaviors

using namespace endstone::nbt;

// Explicit typing & replacement
Tag t = CompoundTag{ { "Damage", ShortTag{5} } };
t["Damage"] = IntTag{6};     // ok (replaces Short with Int)
t["Damage"] = ShortTag{6};   // ok

// Value wrappers read-only in place (C++)
const StringTag& nm = t["Name"].get<StringTag>(); // const ref
std::string s t["Name"].get<StringTag>();         // read

// Typed arrays: braced init and mutation
Tag a = CompoundTag{ { "ba", ByteArrayTag{1,2,3} } };
auto& ba = a["ba"].get<ByteArrayTag>();
ba.push_back(42);
ba[0] = 7;

// Lists: growth & retro-fill
Tag lst = ListTag{};
lst[3];                       // pads with End placeholders
lst[3] = DoubleTag{1.0};      // fixes element type to Double; retro-fills to 0.0

// No implicit empty compound
t["Test"] = {};               // not allowed
t["Test"] = CompoundTag{};    // ok

// Equality
assert(ByteTag{5} == ByteTag{5});
assert(ByteArrayTag{1,2,3} == ByteArrayTag{1,2,3});
assert(CompoundTag{ {"k", IntTag{1}} } == CompoundTag{ {"k", IntTag{1}} });

15 Glossary

  • ValueTag<T> — templated scalar wrapper, read-only in place, implicit conversion to T.
  • ArrayTag<T> — templated typed-array container with STL-like APIs and braced initialisers; aliases ByteArrayTag, IntArrayTag.
  • is_mutable_tag_v<T> — trait indicating container/mutable wrappers (true for ListTag, CompoundTag, ByteArrayTag, IntArrayTag).
  • std::monostate — sentinel representing the End tag in Tag::Storage.
  • Self-shaping — empty Tag becomes List/Compound on first indexed/keyed write.
  • Retro-fill — earlier End placeholders in lists replaced with type defaults once fixed.
  • SNBT — textual NBT; parsed strictly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions