Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -170,12 +170,14 @@ The project provides type-safe builder classes for different environment types:
- Supports `pixi.toml` and `environment.yml` files
- Uses `pixi run --manifest-path <envDir>/pixi.toml` for activation
- Environment structure: `<envDir>/.pixi/envs/default`
- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `pixi.lock` into envDir, installs with `pixi install --frozen`)
- Location: `org.apposed.appose.builder.PixiBuilder`

**MambaBuilder** - Traditional conda environments via micromamba
- Created via `Appose.mamba()` or `Appose.mamba(source)`
- Supports `environment.yml` files
- Uses `mamba run -p <envDir>` for activation
- Does not support lock files (conda/micromamba has no lockfile mechanism)
- Location: `org.apposed.appose.builder.MambaBuilder`

**UvBuilder** - Fast Python virtual environments via uv
Expand All @@ -184,6 +186,7 @@ The project provides type-safe builder classes for different environment types:
- Supports `requirements.txt` files
- Standard Python venv structure (no special activation needed)
- Environment structure: `<envDir>/bin` (or `Scripts` on Windows)
- Lock files: `.lockFile()/.lockContent()/.lockUrl()` for reproducible builds (copies `uv.lock` into envDir, installs with `uv sync --frozen`; requires a `pyproject.toml` declaration)
- Location: `org.apposed.appose.builder.UvBuilder`

**DynamicBuilder** - Auto-detects appropriate builder based on configuration content
Expand All @@ -197,6 +200,7 @@ The project provides type-safe builder classes for different environment types:
- Created via `Appose.custom()` or implicitly via `Appose.system()`
- No package installation; uses whatever executables are on the system
- Methods: `binPaths(paths...)`, `appendSystemPath()`, `inheritRunningJava()`
- Does not support lock files (no package management)
- Location: `org.apposed.appose.builder.SimpleBuilder`

### API Examples
Expand Down Expand Up @@ -235,6 +239,16 @@ Environment env = Appose.file("path/to/environment.yml")
.logDebug()
.build();

// Reproducible build pinned by a lock file (uv: uv.lock -> uv sync --frozen)
Environment env = Appose.uv("path/to/pyproject.toml")
.lockFile("path/to/uv.lock")
.build();

// Reproducible build pinned by a lock file (pixi: pixi.lock -> pixi install --frozen)
Environment env = Appose.pixi("path/to/pixi.toml")
.lockFile("path/to/pixi.lock")
.build();

// Wrap existing environment
Environment env = Appose.wrap("/path/to/existing/env");

Expand Down Expand Up @@ -264,6 +278,29 @@ All builders support subscription methods for monitoring:
- `subscribeError(consumer)` - Error output from build process
- `logDebug()` - Convenience method that logs output and errors to stderr

### Lock Files & Reproducible Builds

`PixiBuilder` and `UvBuilder` support reproducible, lock-file-pinned builds via the
`lockContent(String)`, `lockFile(String|File)`, and `lockUrl(String|URL)` methods (mirroring the
`content`/`file`/`url` declaration API). When a lock is supplied:

- The lock is copied into the environment directory (`uv.lock` / `pixi.lock`) before install.
- The install runs in strict mode so the environment matches the lock exactly, or the build fails:
- **uv**: `uv sync --frozen` (fails if `uv.lock` is missing or out of date with `pyproject.toml`).
Lock files require a `pyproject.toml` declaration (the `requirements.txt` path uses `pip install`
and has no lockfile).
- **pixi**: `pixi install --frozen` (installs the environment exactly as defined in `pixi.lock`,
without re-resolving or updating it — the lock is authoritative). Unlike uv's `--frozen`, pixi's
`--frozen` uses a stale lock as-is rather than failing, but both make builds reproducible.
- The `appose.json` state snapshot records a `lockHash` (SHA-256 of the lock content), so a change to
the lock forces a rebuild via the normal `isUpToDate()` check.
- `DynamicBuilder` (`Appose.file/url/content`) forwards the lock to the detected pixi/uv builder.
- `MambaBuilder` and `SimpleBuilder` do **not** support lock files and throw
`UnsupportedOperationException`.

When no lock is supplied, behavior is unchanged: no strict flag is passed and `appose.json` omits
`lockHash`, so the snapshot is byte-identical to pre-lock-file builds (no spurious rebuilds).

## Related Projects

- appose-python: Python implementation of Appose (https://github.com/apposed/appose-python)
Expand Down
95 changes: 95 additions & 0 deletions src/main/java/org/apposed/appose/Builder.java
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,101 @@ default T url(URL url) throws BuildException {
*/
T scheme(String scheme);

/**
* Specifies lock file content for reproducible builds. When provided, the
* lock file is copied into the environment directory and the install runs in
* strict lock mode so the environment matches the lock (uv: {@code --frozen};
* pixi: {@code --frozen}); the committed lock anchors the build for
* reproducibility. Exact stale-lock handling is tool-specific — see the
* implementing builders.
* <p>
* Not all builders support lock files; builders that do not will throw
* {@link UnsupportedOperationException}. The default implementation throws,
* so only builders that explicitly support locks override this method.
* </p>
*
* @param lockContent Lock file content (e.g., uv.lock, pixi.lock)
* @return This builder instance, for fluent-style programming.
*/
default T lockContent(String lockContent) {
throw new UnsupportedOperationException(
getClass().getSimpleName() + " does not support lock files");
}

/**
* Specifies a lock file path for reproducible builds.
* Reads the file content immediately and delegates to {@link #lockContent(String)}.
*
* @param path Path to the lock file (e.g., "uv.lock", "pixi.lock")
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the file cannot be read
*/
default T lockFile(String path) throws BuildException {
return lockFile(new File(path));
}

/**
* Specifies a lock file for reproducible builds.
* Reads the file content immediately and delegates to {@link #lockContent(String)}.
*
* @param file Lock file (e.g., uv.lock, pixi.lock)
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the file cannot be read
*/
default T lockFile(File file) throws BuildException {
try {
Path filePath = file.toPath();
String fileContent = new String(
Files.readAllBytes(filePath),
StandardCharsets.UTF_8
);
return lockContent(fileContent);
}
catch (IOException e) {
throw new BuildException(this, e);
}
}

/**
* Specifies a URL to fetch lock file content from for reproducible builds.
* Reads the URL content immediately and delegates to {@link #lockContent(String)}.
*
* @param path URL path of the lock file
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the URL cannot be read or is invalid
*/
default T lockUrl(String path) throws BuildException {
try {
return lockUrl(new URL(path));
}
catch (MalformedURLException e) {
throw new BuildException(this, e);
}
}

/**
* Specifies a URL to fetch lock file content from for reproducible builds.
* Reads the URL content immediately and delegates to {@link #lockContent(String)}.
*
* @param url URL to the lock file
* @return This builder instance, for fluent-style programming.
* @throws BuildException If the URL cannot be read
*/
default T lockUrl(URL url) throws BuildException {
try (InputStream stream = url.openStream()) {
ByteArrayOutputStream result = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int length;
while ((length = stream.read(buffer)) != -1) {
result.write(buffer, 0, length);
}
return lockContent(result.toString(StandardCharsets.UTF_8.name()));
}
catch (IOException e) {
throw new BuildException(this, e);
}
}

/**
* Registers a callback method to be invoked when progress happens during environment building.
*
Expand Down
43 changes: 43 additions & 0 deletions src/main/java/org/apposed/appose/builder/BaseBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,8 @@
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.LinkedHashMap;
Expand Down Expand Up @@ -72,6 +74,9 @@ public abstract class BaseBuilder<T extends BaseBuilder<T>> implements Builder<T
/** Configuration file content. */
protected String content;

/** Lock file content for reproducible builds (e.g., uv.lock, pixi.lock), or null when unset. */
protected String lockContent;

/** Explicit scheme (e.g., "pixi.toml", "environment.yml"). */
protected Scheme scheme;

Expand Down Expand Up @@ -132,6 +137,12 @@ public T content(String content) {
return typedThis();
}

@Override
public T lockContent(String lockContent) {
this.lockContent = lockContent;
return typedThis();
}

@Override
public T scheme(String scheme) {
this.scheme = Schemes.fromName(scheme);
Expand Down Expand Up @@ -182,6 +193,38 @@ protected void addStateFields(Map<String, Object> state) {
state.put("channels", channels);
state.put("flags", flags);
state.put("envVars", new TreeMap<>(envVars));
// Record a hash of the lock file content so that lock changes trigger a
// rebuild via the exact-match isUpToDate() comparison. Only added when a
// lock is supplied, so lock-less builds produce a byte-identical
// appose.json (backward compatibility).
if (lockContent != null) state.put("lockHash", computeLockHash(lockContent));
}

/**
* Computes a SHA-256 hash (lowercase hex) of the given content, or null if
* the content is null. Used to snapshot lock files into {@code appose.json}
* without storing the (potentially large) lock content verbatim.
*
* @param content The content to hash (e.g., lock file content).
* @return The SHA-256 hex hash, or null if content is null.
*/
protected static String computeLockHash(String content) {
if (content == null) return null;
try {
MessageDigest md = MessageDigest.getInstance("SHA-256");
byte[] digest = md.digest(content.getBytes(StandardCharsets.UTF_8));
char[] hex = new char[digest.length * 2];
for (int i = 0; i < digest.length; i++) {
int v = digest[i] & 0xff;
hex[i * 2] = Character.forDigit(v >>> 4, 16);
hex[i * 2 + 1] = Character.forDigit(v & 0x0f, 16);
}
return new String(hex);
}
catch (NoSuchAlgorithmException e) {
// SHA-256 is mandated by the JVM specification; this never happens.
throw new RuntimeException("SHA-256 algorithm not available", e);
}
}

/**
Expand Down
3 changes: 3 additions & 0 deletions src/main/java/org/apposed/appose/builder/DynamicBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ private void copyConfigToDelegate(Builder<?> delegate) {
if (envDir != null) delegate.base(envDir);
if (content != null) delegate.content(content);
if (scheme != null) delegate.scheme(scheme.name());
// Forward the lock file if one was provided. Builders that do not
// support locks (mamba/custom) override lockContent() to throw.
if (lockContent != null) delegate.lockContent(lockContent);
delegate.channels(channels);
progressSubscribers.forEach(delegate::subscribeProgress);
outputSubscribers.forEach(delegate::subscribeOutput);
Expand Down
7 changes: 7 additions & 0 deletions src/main/java/org/apposed/appose/builder/MambaBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,13 @@ public String envType() {
return "mamba";
}

@Override
public MambaBuilder lockContent(String lockContent) {
throw new UnsupportedOperationException(
"MambaBuilder does not support lock files; " +
"conda/micromamba environments have no lockfile mechanism.");
}

@Override
public Environment build() throws BuildException {
File envDir = resolveEnvDir();
Expand Down
48 changes: 44 additions & 4 deletions src/main/java/org/apposed/appose/builder/PixiBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,22 @@ public Environment build() throws BuildException {
}
}

// Validate lock-file compatibility. pixi lockfiles apply to manifest-
// based builds (pixi.toml / pyproject.toml); programmatic builds and
// imported environment.yml have no user manifest to lock against.
if (lockContent != null) {
if (content == null) {
throw new IllegalArgumentException(
"PixiBuilder lock files require a declaration file via .file()/.content(); " +
"programmatic builds cannot be locked.");
}
if (!"pixi.toml".equals(scheme.name()) && !"pyproject.toml".equals(scheme.name())) {
throw new IllegalArgumentException(
"PixiBuilder lock files require a pixi.toml or pyproject.toml declaration; " +
"environment.yml imports have no lockfile mechanism.");
}
}

Pixi pixi = new Pixi();

// Set up progress/output consumers.
Expand Down Expand Up @@ -160,6 +176,13 @@ else if ("environment.yml".equals(scheme.name())) {
pixi.exec("init", "--import", environmentYamlFile.getAbsolutePath(), envDir.getAbsolutePath());
}

// If a lock file was provided, copy it into the env dir so the
// subsequent install runs strictly from it (--frozen).
if (lockContent != null) {
File pixiLockFile = new File(envDir, "pixi.lock");
Files.write(pixiLockFile.toPath(), lockContent.getBytes(StandardCharsets.UTF_8));
}

// Add any programmatic channels to augment source file.
if (!channels.isEmpty()) {
pixi.addChannels(envDir, channels.toArray(new String[0]));
Expand Down Expand Up @@ -213,7 +236,7 @@ else if ("environment.yml".equals(scheme.name())) {
}
}

runPixiInstall(pixi, envDir);
runPixiInstall(pixi, envDir, lockContent != null);
writeApposeStateFile(envDir);
return buildPixiEnvironment(pixi, envDir);
}
Expand Down Expand Up @@ -242,6 +265,12 @@ public Environment wrap(File envDir) throws BuildException {
scheme = Schemes.fromName("pyproject.toml");
}
}
// If a pixi.lock is present, capture it too so rebuild() reproduces
// the exact locked environment even after the directory is deleted.
File pixiLock = new File(envDir, "pixi.lock");
if (pixiLock.exists() && pixiLock.isFile()) {
lockContent = new String(Files.readAllBytes(pixiLock.toPath()), StandardCharsets.UTF_8);
}
}
catch (IOException e) {
throw new BuildException(this, e);
Expand All @@ -261,7 +290,7 @@ private static List<String> withFlag(List<String> flags, String flag) {
return result;
}

private void runPixiInstall(Pixi pixi, File envDir) throws IOException, InterruptedException {
private void runPixiInstall(Pixi pixi, File envDir, boolean frozen) throws IOException, InterruptedException {
File manifestFile = new File(envDir, "pyproject.toml");
if (!manifestFile.exists()) manifestFile = new File(envDir, "pixi.toml");

Expand All @@ -278,9 +307,20 @@ private void runPixiInstall(Pixi pixi, File envDir) throws IOException, Interrup
pixi.setErrorConsumer(monitor::intercept);
}

// Ensure the pixi environment is fully installed.
// Ensure the pixi environment is fully installed. When a lock was
// provided, pass --frozen so pixi installs the environment exactly as
// defined in pixi.lock, without re-resolving or updating it. This makes
// the build reproducible: the installed environment always matches the
// committed lock, byte for byte. (Note: pixi's --frozen, unlike uv's,
// uses the lock as-is even when the manifest has drifted -- the lock is
// authoritative, which is the reproducibility contract.)
try {
pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath());
if (frozen) {
pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath(), "--frozen");
}
else {
pixi.exec("install", "--manifest-path", manifestFile.getAbsolutePath());
}
}
finally {
if (monitor != null) {
Expand Down
7 changes: 7 additions & 0 deletions src/main/java/org/apposed/appose/builder/SimpleBuilder.java
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,13 @@ public SimpleBuilder channels(List<String> channels) {
"It uses existing executables without package management.");
}

@Override
public SimpleBuilder lockContent(String lockContent) {
throw new UnsupportedOperationException(
"SimpleBuilder does not support lock files. " +
"Custom environments use existing executables without package management.");
}

// -- Internal methods --

@Override
Expand Down
Loading