Skip to content

Fix instaparse macro loading by excluding .clj files from relocation - #14

Open
hellolittlej wants to merge 1 commit into
mainfrom
fix-macros-load
Open

Fix instaparse macro loading by excluding .clj files from relocation#14
hellolittlej wants to merge 1 commit into
mainfrom
fix-macros-load

Conversation

@hellolittlej

@hellolittlej hellolittlej commented May 13, 2026

Copy link
Copy Markdown

Context

The mql-jvm module produces a shadow JAR — a fat JAR that bundles and relocates its Clojure dependencies under the io.mantisrx.mql.shaded.* namespace to
avoid classpath conflicts. The issue is that Clojure .clj/.cljc source files were being included in the shaded JAR along with the compiled .class files.

This causes a subtle but serious problem: instaparse/macros.clj (and other Clojure source files) gets relocated into the shaded namespace path. When
Clojure's macro system tries to load instaparse/macros.clj at runtime, it looks for the file under its original path — but the shadow plugin has moved it to
io/mantisrx/mql/shaded/instaparse/macros.clj. This breaks macro loading.

We have a user putting depends on the mql library in their spark job and failed at runtime due to FileNotFoundException

Caused by: java.io.FileNotFoundException: Could not locate instaparse/macros__init.class or instaparse/macros.clj on classpath., compiling:(io/mantisrx/mql/shaded/instaparse/core.cljc:1:1)
	at io.mantisrx.mql.shaded.clojure.lang.Compiler.load(Compiler.java:7391)
	at io.mantisrx.mql.shaded.clojure.lang.RT.loadResourceScript(RT.java:372)
	at io.mantisrx.mql.shaded.clojure.lang.RT.loadResourceScript(RT.java:363)
	at io.mantisrx.mql.shaded.clojure.lang.RT.load(RT.java:453)
	at io.mantisrx.mql.shaded.clojure.lang.RT.load(RT.java:419)
	at io.mantisrx.mql.shaded.clojure.core$load$fn__5677.invoke(core.clj:5893)
	at io.mantisrx.mql.shaded.clojure.core$load.invokeStatic(core.clj:5892)
	at io.mantisrx.mql.shaded.clojure.core$load.doInvoke(core.clj:5876)
	at io.mantisrx.mql.shaded.clojure.lang.RestFn.invoke(RestFn.java:408)
	at io.mantisrx.mql.shaded.clojure.core$load_one.invokeStatic(core.clj:5697)
	at io.mantisrx.mql.shaded.clojure.core$load_one.invoke(core.clj:5692)
	at io.mantisrx.mql.shaded.clojure.core$load_lib$fn__5626.invoke(core.clj:5737)
	at io.mantisrx.mql.shaded.clojure.core$load_lib.invokeStatic(core.clj:5736)
	at io.mantisrx.mql.shaded.clojure.core$load_lib.doInvoke(core.clj:5717)
	at io.mantisrx.mql.shaded.clojure.lang.RestFn.applyTo(RestFn.java:142)
	at io.mantisrx.mql.shaded.clojure.core$apply.invokeStatic(core.clj:648)
	at io.mantisrx.mql.shaded.clojure.core$load_libs.invokeStatic(core.clj:5774)
	at io.mantisrx.mql.shaded.clojure.core$load_libs.doInvoke(core.clj:5758)
	at io.mantisrx.mql.shaded.clojure.lang.RestFn.applyTo(RestFn.java:137)
	at io.mantisrx.mql.shaded.clojure.core$apply.invokeStatic(core.clj:648)
	at io.mantisrx.mql.shaded.clojure.core$require.invokeStatic(core.clj:5796)
	at io.mantisrx.mql.shaded.clojure.core$require.doInvoke(core.clj:5796)
	at io.mantisrx.mql.shaded.clojure.lang.RestFn.invoke(RestFn.java:512)
	at io.mantisrx.mql.core$loading__5569__auto____67.invoke(core.cljc:1)
	at io.mantisrx.mql.core__init.load(Unknown Source)

I'm putting ^ into the library so we don't leak the source files to each end consumers

Checklist

  • ./gradlew build compiles code correctly
  • Added new tests where applicable
  • ./gradlew test passes all tests
  • Extended README or added javadocs where applicable
  • Added copyright headers for new files from CONTRIBUTING.md

@Andyz26

Andyz26 commented May 18, 2026

Copy link
Copy Markdown
Contributor

pause/close this for now?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants