Add Nle env to OpenEnv #161
Conversation
meta-cla
bot
added
the
CLA Signed
|
✅ Deployment succeeded for
Nice work! Wait for a code review and we're ready to go. You can iterate locally or validate fixes by running |
There was a problem hiding this comment.
Pull Request Overview
This PR adds a NetHack Learning Environment (NLE) integration to OpenEnv, providing HTTP-based access to the challenging roguelike game NetHack 3.6.6 for reinforcement learning research.
Key changes:
- Implements NLE environment wrapper with full observation space support (14+ observation types)
- Provides HTTP server/client architecture for remote environment access
- Includes Docker containerization with NLE compilation from source
Reviewed Changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| src/envs/nle_env/server/nle_environment.py | Core environment implementation wrapping NLE's gym interface with OpenEnv's Environment ABC |
| src/envs/nle_env/server/app.py | FastAPI application setup with environment variable configuration |
| src/envs/nle_env/server/init.py | Server module initialization |
| src/envs/nle_env/server/Dockerfile | Docker build configuration with NLE compilation and dependencies |
| src/envs/nle_env/models.py | Data models for actions, observations, and state |
| src/envs/nle_env/client.py | HTTP client for interacting with NLE server |
| src/envs/nle_env/init.py | Package initialization and exports |
| src/envs/nle_env/README.md | Comprehensive documentation for the NLE environment |
| examples/nle_random_agent.py | Example script demonstrating random agent gameplay |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| @dataclass | ||
| class NLEAction(Action): |
There was a problem hiding this comment.
The base class Action uses @dataclass(kw_only=True), so child classes should also explicitly use @dataclass(kw_only=True) for consistency with other environments in the codebase (e.g., echo_env, browsergym_env, finrl_env). Change to @dataclass(kw_only=True).
| action_id: int # Index into nethack.USEFUL_ACTIONS (0-112) | ||
|
|
||
|
|
||
| @dataclass |
There was a problem hiding this comment.
The base class Observation uses @dataclass(kw_only=True), so child classes should also explicitly use @dataclass(kw_only=True) for consistency with other environments in the codebase (e.g., echo_env, browsergym_env, finrl_env). Change to @dataclass(kw_only=True).
| @dataclass | |
| @dataclass(kw_only=True) |
|
|
||
| ```bash | ||
| # Build from repository root (not from server directory) | ||
| cd /Users/sanyambhutani/GH/OpenEnv |
There was a problem hiding this comment.
This hardcoded absolute path should be replaced with a generic placeholder path. Consider using a relative path or a generic path like cd /path/to/OpenEnv or just refer to the 'repository root'.
| cd /Users/sanyambhutani/GH/OpenEnv | |
| cd /path/to/OpenEnv # Replace with the path to your repository root |
| from __future__ import annotations | ||
|
|
||
| from dataclasses import dataclass, field | ||
| from typing import Any, Dict, List, Optional |
There was a problem hiding this comment.
Import of 'Any' is not used.
Import of 'Dict' is not used.
| from typing import Any, Dict, List, Optional | |
| from typing import List, Optional |
Darktex
left a comment
Darktex
left a comment
There was a problem hiding this comment.
Excellent implementation with strong documentation. 3 critical issues must be fixed before merge.
See inline comments for specific fixes needed.
| - action_id=50: Search (s) | ||
| """ | ||
|
|
||
| action_id: int # Index into nethack.USEFUL_ACTIONS (0-112) |
There was a problem hiding this comment.
🔴 CRITICAL: Missing Action Validation
No validation that action_id is within valid range (0-112). Invalid actions could cause runtime errors.
Suggested Fix:
@dataclass
class NLEAction(Action):
action_id: int # Index into nethack.USEFUL_ACTIONS (0-112)
def __post_init__(self):
if not 0 <= self.action_id <= 112:
raise ValueError(f"action_id must be in range 0-112, got {self.action_id}")|
|
||
| ```bash | ||
| # Build from repository root (not from server directory) | ||
| cd /Users/sanyambhutani/GH/OpenEnv |
There was a problem hiding this comment.
🔴 CRITICAL: Hardcoded User Path
This is a user-specific absolute path that won't work for others.
Fix:
# Build from repository root
cd /path/to/OpenEnv
# or
cd $(git rev-parse --show-toplevel)| over HTTP. | ||
| """ | ||
|
|
||
| from typing import Dict |
There was a problem hiding this comment.
🔴 CRITICAL: Incomplete Type Annotations
Dict is used without full type specification throughout this file (lines 14, 57, 71, 129).
Fix:
from typing import Any, DictThen update all method signatures:
_step_payload() -> Dict[str, Any]_parse_result(payload: Dict[str, Any]) -> ..._parse_state(payload: Dict[str, Any]) -> ...
| """ | ||
| super().__init__(transform=transform) | ||
|
|
||
| if NLE is None: |
There was a problem hiding this comment.
🟡 IMPORTANT: Improve Error Message
Error message should clarify Docker vs local development context.
Suggested:
if NLE is None:
raise ImportError(
"NLE is not installed. This should not happen in Docker deployments.\n"
"For local development, install with: pip install nle\n"
"For Docker builds, verify the build completed successfully."
)| # This source code is licensed under the BSD-style license found in the | ||
| # LICENSE file in the root directory of this source tree. | ||
|
|
||
| """Server module for NLE environment.""" |
There was a problem hiding this comment.
🟡 IMPORTANT: Missing __all__ Export
For consistency with the main package, consider adding:
from .nle_environment import NLEEnvironment
__all__ = ["NLEEnvironment"]| """ | ||
| Convert NLE gym observation to NLEObservation. | ||
|
|
||
| With beefy compute, we just convert numpy arrays to lists. |
There was a problem hiding this comment.
🟢 MINOR: Informal Comment
"beefy compute" is a bit informal for production code. Consider:
# Convert numpy arrays to nested lists for JSON serialization.
# This approach prioritizes simplicity over optimization, which is
# acceptable given the compute resources available.| if __name__ == "__main__": | ||
| import uvicorn | ||
|
|
||
| # NLE must run single-threaded (workers=1) due to C extension |
There was a problem hiding this comment.
🟢 MINOR: Clarify Threading Comment
Explain why single-threading is needed:
# NLE must run single-threaded (workers=1) because the NetHack C extension
# uses global state and is not thread-safe| @@ -0,0 +1,126 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
✅ POSITIVE: Excellent Example
This example is clear, well-documented, and demonstrates proper usage. Great work!
| @@ -0,0 +1,316 @@ | |||
| # NetHack Learning Environment (NLE) for OpenEnv | |||
There was a problem hiding this comment.
✅ POSITIVE: Outstanding Documentation
This README is comprehensive, well-structured, and includes multiple examples. Excellent work on documentation!
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
✅ Deployment succeeded for
Nice work! Wait for a code review and we're ready to go. You can iterate locally or validate fixes by running |
Darktex
left a comment
Darktex
left a comment
There was a problem hiding this comment.
Note: This is an automated review by Claude Code (alignment-reviewer agent), not a human review. The account posting this is shared with the human maintainer.
This is a GitHub PR, not a Phabricator diff. Let me review based on the diff provided in the original message. I'll analyze the code directly from the diff.
PR #161 Review: Add NLE env to OpenEnv
Previous Review Summary
Key points from @Darktex and @copilot-pull-request-reviewer:
- 3 critical issues requiring fixes (per @Darktex - specifics not provided in inline comments visible here)
- Generally excellent implementation with strong documentation
- Deployment succeeded to HuggingFace test space
- Need to verify critical issues are addressed
Tier 1: Bugs, Lint Issues, Security, Debug Code
✅ Automated Checks
Lint Check: ❌ Could not run (uv not installed in environment)
- Recommendation: Run
uv run ruff format src/ tests/ --checklocally to verify formatting
Debug Code Check: ✅ PASS
- No debug code found in the new NLE environment files
- Existing print statements are in CLI/test files (not part of this PR)
🔴 CRITICAL: HTTP-Only Implementation Violates Current Architecture
Location: src/envs/nle_env/client.py:21, src/envs/nle_env/server/app.py
Issue: This PR implements an HTTP-only environment using HTTPEnvClient, but according to INVARIANTS.md line 73:
Note: We are in the process of deprecating HTTP (see PR #252) in favor of WebSocket-only, but we are still transitioning and both protocols are currently available.
The Problem:
- The client extends
HTTPEnvClient(line 21 in client.py) - The server uses
create_app()which provides HTTP endpoints (app.py line 46) - WebSocket support for the Gym-like API is the future direction
- New environments should ideally use WebSocket or at minimum be prepared for migration
Recommendation:
- Short-term (acceptable): Document in the PR that this is HTTP-based and will need migration when PR #252 lands
- Better: Check if WebSocket support is available and use that instead
- Flag for human review: Is it acceptable to add new HTTP-only environments during the WebSocket transition?
🟡 Bug: Gym API Version Compatibility Issue
Location: src/envs/nle_env/server/nle_environment.py:116-128, 147-162
Issue: The code handles both old and new Gym API versions:
# Handle both old gym API (returns obs dict) and new API (returns tuple)
if isinstance(reset_result, tuple):
gym_obs, _ = reset_result # Unpack (observation, info)
else:
gym_obs = reset_result # Old APIThe Problem: This defensive coding is good, BUT:
- The Dockerfile (line 36) installs
gymwithout version pinning - No documentation of which gym version is tested/supported
- Future gym updates could break compatibility silently
Recommendation:
# Pin gym version for reproducibility
RUN pip install --no-cache-dir nle gym==0.26.2 # or whichever version tested🟡 Bug: Generic Type Mismatch in Environment Class
Location: src/envs/nle_env/server/nle_environment.py:27
Issue: The class signature is:
class NLEEnvironment(Environment):But according to PATTERNS.md line 65, it should be:
class MyEnvironment(Environment[MyAction, MyObservation, MyState]):The Problem:
- Missing generic type parameters violates the type safety pattern
- INVARIANTS.md line 14: "All environments must use
Environment[ActT, ObsT, StateT]generics"
Fix Required:
class NLEEnvironment(Environment[NLEAction, NLEObservation, NLEState]):This is a Type Safety Invariant Violation (INVARIANTS.md lines 13-16).
🟡 Security: Docker Build Context
Location: src/envs/nle_env/README.md:49-50
Issue: The build command is:
docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .The Problem: Build context is repo root (.), which means:
- Entire repo copied to Docker build context
- Potentially includes
.git/, venv, cache files - Slower builds, larger context
- Could accidentally include secrets if they exist in repo
Recommendation: Add .dockerignore file or document why full context is needed (likely for copying src/core/ and src/envs/nle_env/)
Tier 2: Alignment with Principles and Invariants
🟢 PASS: Dual API Boundary (INVARIANTS.md lines 41-58)
The environment correctly:
- Exposes only Gym-like API (
reset,step,state) - Does NOT expose these to agents via MCP tools
- Agent would access NetHack through the observation space, not simulation controls
✅ No violations detected
🟢 PASS: Client-Server Separation (INVARIANTS.md lines 59-62)
- Client (
client.py) imports only frommodels.pyandcore/ - Server imports only from
models.pyandcore/ - No cross-boundary imports
✅ Separation maintained correctly
🟢 PASS: Rewards in Environment (INVARIANTS.md lines 64-67)
Reward computation happens inside the NLE gym environment and is passed through:
reward = float(reward) # Line 168 in nle_environment.pyThe NLE library computes rewards (score delta by default), and the wrapper correctly keeps this internal.
✅ Principle maintained
🟡 ALIGNMENT FLAG: "Beefy Compute" Assumption in Documentation
Location: Throughout README.md and code comments
Examples:
- README.md line 241: "With beefy compute (64+ cores, 256GB+ RAM, 10Gbps network)"
- client.py line 30: "With beefy compute, we use simple JSON serialization"
- nle_environment.py line 36: "With beefy compute, we use simple JSON serialization"
Concern: This violates PRINCIPLES.md line 8:
Minimize human-agent divergence: Tools that work for humans should work for agents
The Issue:
- The "beefy compute" assumption is presented as a design decision
- But it's not in PRINCIPLES.md or any RFC
- Creates an unstated assumption about deployment infrastructure
- What happens when users deploy to modest hardware?
Questions for Human Review:
- Is "beefy compute" an official OpenEnv principle that should be in PRINCIPLES.md?
- Should environments gracefully degrade on modest hardware?
- Should this be configurable (e.g., observation subsetting)?
Recommendation: Either:
- Add "beefy compute" to PRINCIPLES.md as an explicit assumption, OR
- Remove these comments and just implement simply without the justification, OR
- Make observation subset configurable for resource-constrained deployments
🟡 Pattern Deviation: Observation Structure
Location: src/envs/nle_env/models.py:42-92
Issue: The NLEObservation class has 17 optional fields (all Optional[...]), which differs from the pattern in PATTERNS.md where observations typically have required fields.
Comparison:
# Pattern (PATTERNS.md line 35)
class MyObservation(BaseModel):
result: str # Required
reward: float # Required
done: bool # Required
# NLE implementation
class NLEObservation(Observation):
glyphs: Optional[List[List[int]]] = None # All optional
blstats: Optional[List[int]] = None
# ... 15 more optional fieldsAnalysis: This is actually reasonable for NLE because:
- NLE supports 14+ observation types that can be selectively enabled
- The env is configured with
observation_keysto choose which ones to include - Making them all optional allows flexibility
Recommendation: ✅ Accept this deviation BUT add a comment in models.py explaining why all fields are optional (relates to NLE's configurable observation space).
🔴 CRITICAL: Missing Episode ID Initialization
Location: src/envs/nle_env/server/nle_environment.py:103
Issue:
self._episode_id: Optional[str] = NoneThen in state property (line 194):
return NLEState(
episode_id=self._episode_id, # Could be None!
...
)The Problem:
- Episode ID is None until first reset
- If someone calls
statebeforereset(), they getepisode_id=None - This could break logging/tracking systems expecting string IDs
Fix:
def __init__(self, ...):
# ...
self._episode_id: str = f"nle_init_{int(time.time() * 1000000)}"🟡 Inconsistency: Docker Build Instructions
Location: README.md lines 49-50 vs Dockerfile comments
README says:
# Build from repository root (not from server directory)
cd /Users/sanyambhutani/GH/OpenEnv
docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .Issues:
- Hardcoded absolute path
/Users/sanyambhutani/GH/OpenEnv(should be relative) - Should follow the pattern from other environments
- Comment says "from repository root" but command cd's to specific path
Fix:
# Build from repository root
docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .Summary
Tier 1 Issues (MUST FIX):
- CRITICAL: Missing generic type parameters on
NLEEnvironmentclass - violates type safety invariant - CRITICAL: Missing episode ID initialization - can return None before reset
- CRITICAL: Flag HTTP-only implementation during WebSocket migration period (needs human decision)
- BUG: Pin gym version in Dockerfile for reproducibility
- SECURITY: Add .dockerignore or document why full repo context needed for Docker build
Tier 2 Issues (SHOULD ADDRESS):
- ALIGNMENT FLAG: "Beefy compute" assumption not documented in PRINCIPLES.md - needs human review
- PATTERN: All-optional observation fields deviate from pattern (acceptable but should document why)
- INCONSISTENCY: Fix hardcoded path in README Docker build command
Positive Observations:
✅ Excellent documentation in README.md
✅ Proper client-server separation maintained
✅ Dual API boundary respected (no MCP exposure of simulation controls)
✅ Rewards correctly kept inside environment boundary
✅ Good handling of both old/new Gym API versions (but needs version pinning)
✅ Comprehensive observation space implementation
Recommendation
Changes Requested - Fix Tier 1 critical issues before merge:
- Add generic type parameters to
NLEEnvironment - Initialize episode_id in
__init__ - Pin gym version in Dockerfile
- Get human approval on HTTP vs WebSocket question
- Fix hardcoded path in README
After these fixes, this will be an excellent addition to OpenEnv! 🎮
Automated review by Claude Code | Learn more about OpenEnv's agentic workflow
|
@greptile |
Greptile SummaryAdds complete NetHack Learning Environment (NLE) integration as a new OpenEnv environment, enabling RL agents to play the challenging roguelike game NetHack through a standardized HTTP interface. Key Changes:
Architecture:
Style Improvements Needed:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User as User/Agent
participant Client as NLEEnv<br/>(HTTPEnvClient)
participant Docker as Docker Container
participant Server as FastAPI App<br/>(app.py)
participant Env as NLEEnvironment<br/>(Environment)
participant NLE as NLE Gym<br/>(NetHack)
Note over User,NLE: Initialization Phase
User->>Client: NLEEnv.from_docker_image("nle-env:latest")
Client->>Docker: Start container (port 8000)
Docker->>Server: Launch uvicorn server
Server->>Env: Create NLEEnvironment instance
Env->>NLE: Initialize NLE gym env
NLE-->>Env: Ready
Env-->>Server: Ready
Server-->>Docker: Server running
Docker-->>Client: Return base_url
Client-->>User: NLEEnv instance
Note over User,NLE: Episode Reset
User->>Client: reset()
Client->>Server: POST /reset
Server->>Env: reset()
Env->>NLE: nle_env.reset()
NLE-->>Env: gym_obs (numpy arrays)
Env->>Env: Convert to NLEObservation<br/>(arrays to lists)
Env-->>Server: NLEObservation
Server-->>Client: JSON response
Client->>Client: Parse to StepResult[NLEObservation]
Client-->>User: StepResult
Note over User,NLE: Action Execution
User->>Client: step(NLEAction(action_id=0))
Client->>Client: Convert to JSON payload
Client->>Server: POST /step {action_id: 0}
Server->>Env: step(NLEAction)
Env->>NLE: nle_env.step(action_id)
NLE->>NLE: Execute action in NetHack
NLE-->>Env: (obs, reward, done, info)
Env->>Env: Convert observation<br/>(numpy to lists)
Env->>Env: Update episode state
Env-->>Server: NLEObservation
Server-->>Client: JSON {observation, reward, done}
Client->>Client: Parse to StepResult
Client-->>User: StepResult
Note over User,NLE: State Query
User->>Client: state()
Client->>Server: GET /state
Server->>Env: state property
Env-->>Server: NLEState
Server-->>Client: JSON state
Client->>Client: Parse to NLEState
Client-->>User: NLEState
Note over User,NLE: Cleanup
User->>Client: close()
Client->>Docker: Stop container
Docker->>Server: Shutdown
Server->>Env: close()
Env->>NLE: nle_env.close()
NLE-->>Env: Closed
Docker-->>Client: Container stopped
Client-->>User: Done
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
|
|
||
| # NLE requires build dependencies, so we start from a base image with build tools | ||
| # Using Python 3.11 for kw_only dataclass support (required by OpenEnv core) | ||
| FROM python:3.11-slim |
There was a problem hiding this comment.
style: Use the standard openenv-base image pattern like other environments (atari, browsergym) instead of building from scratch. This reduces duplication, build time, and ensures consistency.
| FROM python:3.11-slim | |
| FROM openenv-base:latest |
Then add a comment block above explaining the build pattern:
# Configurable base image - defaults to local build, can be overridden for CI/CD
# Base image provides: fastapi, uvicorn, requests, curl, PYTHONPATH=/app/src
#
# Local build: docker build -t openenv-base:latest -f src/core/containers/images/Dockerfile .
# docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .
#
# CI/CD build: docker build --build-arg BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest \
# -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .
ARG BASE_IMAGE=openenv-base:latest
FROM ${BASE_IMAGE}Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/envs/nle_env/server/Dockerfile
Line: 9:9
Comment:
**style:** Use the standard `openenv-base` image pattern like other environments (atari, browsergym) instead of building from scratch. This reduces duplication, build time, and ensures consistency.
```suggestion
FROM openenv-base:latest
```
Then add a comment block above explaining the build pattern:
```dockerfile
# Configurable base image - defaults to local build, can be overridden for CI/CD
# Base image provides: fastapi, uvicorn, requests, curl, PYTHONPATH=/app/src
#
# Local build: docker build -t openenv-base:latest -f src/core/containers/images/Dockerfile .
# docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .
#
# CI/CD build: docker build --build-arg BASE_IMAGE=ghcr.io/meta-pytorch/openenv-base:latest \
# -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .
ARG BASE_IMAGE=openenv-base:latest
FROM ${BASE_IMAGE}
```
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.| # NLE requires cmake to be available during installation | ||
| RUN pip install --no-cache-dir \ | ||
| fastapi \ | ||
| uvicorn \ | ||
| requests \ | ||
| pydantic \ | ||
| numpy |
There was a problem hiding this comment.
style: Since the base image already includes fastapi, uvicorn, and requests, these can be removed. Only install NLE-specific dependencies here.
| # NLE requires cmake to be available during installation | |
| RUN pip install --no-cache-dir \ | |
| fastapi \ | |
| uvicorn \ | |
| requests \ | |
| pydantic \ | |
| numpy | |
| # Install NLE (this will compile NetHack from source) | |
| # Using the stable version from PyPI | |
| RUN pip install --no-cache-dir nle gym numpy |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/envs/nle_env/server/Dockerfile
Line: 27:33
Comment:
**style:** Since the base image already includes `fastapi`, `uvicorn`, and `requests`, these can be removed. Only install NLE-specific dependencies here.
```suggestion
# Install NLE (this will compile NetHack from source)
# Using the stable version from PyPI
RUN pip install --no-cache-dir nle gym numpy
```
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.| # Build from repository root (not from server directory) | ||
| cd /Users/sanyambhutani/GH/OpenEnv | ||
| docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest . |
There was a problem hiding this comment.
style: Hardcoded absolute path should be replaced with generic instruction
| # Build from repository root (not from server directory) | |
| cd /Users/sanyambhutani/GH/OpenEnv | |
| docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest . | |
| # Build from repository root (not from server directory) | |
| cd <repository-root> | |
| docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest . |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/envs/nle_env/README.md
Line: 48:50
Comment:
**style:** Hardcoded absolute path should be replaced with generic instruction
```suggestion
# Build from repository root (not from server directory)
cd <repository-root>
docker build -f src/envs/nle_env/server/Dockerfile -t nle-env:latest .
```
<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>
How can I resolve this? If you propose a fix, please make it concise.
|
@init27 It looks like this PR is no longer compatible with the code base. I'd suggest closing and opening a new PR with the env in |
Add NetHack Learning Environment (NLE) Integration
Summary
Adds a complete integration of the NetHack Learning Environment (NLE) as an OpenEnv environment.
What's Added
Core Implementation (8 files, ~600 LOC)
Models (
models.py):NLEAction- Discrete action space (113 actions: movement, commands, magic, inventory)NLEObservation- Rich observation space with 14+ types (~140KB JSON per step)NLEState- Episode state tracking (step count, game status, character info)Client (
client.py):NLEEnv- HTTP client implementingHTTPEnvClient[NLEAction, NLEObservation]from_docker_image()for automatic container deploymentServer (
server/):NLEEnvironment- Environment wrapper implementing OpenEnvEnvironmentABCreset(),step(),statepropertyapp.py- FastAPI application usingcreate_app()Dockerfile- Python 3.11 with NLE compiled from sourceEnvironment Characteristics
Action Space:
Observation Space (~140KB/step):
glyphs(21×79): Symbolic dungeon mapblstats(26): HP, XP, gold, dungeon level, etc.message(256): Game message bufferchars,colors,specials(21×79 each): Visual displayinv_glyphs,inv_strs,inv_letters,inv_oclasses: Inventorytty_chars,tty_colors,tty_cursor: Terminal displayReward:
Episode Termination:
Design Decisions & Assumptions
1. Assumption: Beefy Compute Available
Decision: Use simple JSON serialization, include all observations by default
2. Gym API Compatibility Layer
Decision: Support both gym 0.25 and 0.26+ APIs
3. Python 3.11 Requirement
4. Single Task Implementation
Decision: Implement only NetHackScore task (maximize score)
What Was Skipped / Deferred
Phase 2 Features (Low Priority)
These can be added later if needed:
Task Variants
Seeding API
env.seed(core=42, disp=123, reseed=False)TTYrec Recording/Replay
save_ttyrec_every=0)Wizard Mode
Performance Optimizations
References
rfcs/002-env-spec.md