(clojure.java.classpath/classpath) produces wrong results with plugin 1.3.20

[drlivingston]

I have code that needs to look over the class path. It gets it using:

(clojure.java.classpath/classpath)

Using clojure-maven-plugin 1.3.9 all is well that call gets me quite a large seq of files. With clojure-maven-plugin 1.3.20 I see something like the following and only the following:

(#<File /var/folders/bk/w587c61d02s2hkn5zqs4zlnm0000gs/T/clojuremavenplugin2851089989068089718jar>)

If you force that thing to be a JarFile and then look at what's in it there is one manifest file and that's it.

Kevin

[talios]

On 1 May 2014, at 15:36, Kevin Livingston wrote:

Using clojure-maven-plugin 1.3.9 all is well that call gets me quite a
large seq of files. With clojure-maven-plugin 1.3.20 I see something
like the following and only the following:

Interesting - which goal is that from? I know during those releases we
switched how the VM is spawned - does the manifest file include a class
path setting at all?

I'll take a look sometime tomorrow ( it's 11:48pm now )...

[drlivingston]

I believe the manifest is empty. The problem in 1.3.20 was with both clojure:nrepl and clojure:repl. Things were working in 1.3.9 with repl for sure and I'm pretty sure with swank too. (I was in the process of moving up to nrepl when all this mess started.)

Thanks for looking into it.

[talios]

On 5 May 2014, at 8:18, Kevin Livingston wrote:

I believe the manifest is empty. The problem in 1.3.20 was with both
clojure:nrepl and clojure:repl. Things were working in 1.3.9 with
repl for sure and I'm pretty sure with swank too. (I was in the
process of moving up to nrepl when all this mess started.)

A quick test:

 $ mvn clojure:repl
 [INFO] Scanning for projects...
 [INFO]
 [INFO] Using the builder 

org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder
with a thread count of 1
[INFO]

[INFO]

 [INFO] Building clojure-maven-example 1.2-SNAPSHOT
 [INFO] 

---

 [INFO]
 [INFO] --- clojure-maven-plugin:1.3.20:repl (default-cli) @ 

clojure-maven-example ---
Clojure 1.6.0
user=> (use 'clojure.java.classpath)
nil
user=> (classpath)
(#<File
/private/var/folders/2h/76nn14b502q410szcd74lv200000gn/T/clojuremavenplugin967447757674925251jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/dnsns.jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/localedata.jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/sunec.jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar>
#<File
/Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/jre/lib/ext/zipfs.jar>
#<File /System/Library/Java/Extensions/AppleScriptEngine.jar>
#<File /System/Library/Java/Extensions/dns_sd.jar>
#<File /System/Library/Java/Extensions/j3daudio.jar> #<File
/System/Library/Java/Extensions/j3dcore.jar>
.....

This is using:

 <build>
     <plugins>
         <plugin>
             <groupId>com.theoryinpractise</groupId>
             <artifactId>clojure-maven-plugin</artifactId>
             <version>1.3.20</version>
             <extensions>true</extensions>
         </plugin>
         <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-release-plugin</artifactId>
             <version>2.1</version>
             <configuration>
                 <preparationGoals>clean install</preparationGoals>
                 <goals>deploy</goals>
                 <autoVersionSubmodules>true</autoVersionSubmodules>
                 <pushChanges>false</pushChanges>
                 <localCheckout>true</localCheckout>

true






org.clojure
clojure
1.6.0


org.clojure
java.classpath
0.2.2

Am I doing anything different than you here?

[drlivingston]

So I deleted my previous post. I've been systematically cutting and cutting things. The one thing that can reliably modulate the problem is the following.

I have a project A parent pom, it has a dependency management section with this:

    <dependency>
      <groupId>org.clojure</groupId>
      <artifactId>java.classpath</artifactId>
      <version>0.1.0</version>
    </dependency>

it has a child B that compiles and uses some classpath code.

Then I have project C that depends on B and classpath 0.2.2. In this configuration C when running is blinded to the classpath. Changing the dependency management section in A to use 0.2.2 and C can see the classpath.

If I gut the source code from B that refers to the classpath but leave the dependency then everything functions fine in C regardless of what version I have in A. This is odd because I can't figure out how this is modulating C's view of the common library. If anything it should break B by going to the wrong version, right?

If I call mvn:dependency:tree on C I only see version 0.2.2 no matter what.

In this particular case I happen to have control of A and B so I think I can upgrade them and not mess anyone up. But this is certainly weird.

This seems to happen regardless of clojure-maven-plugin -- the project I was depending on when I was modulating that was also modulating the classpath dependency. Sorry about that. Do you think there is somewhere else I should report this? I guess I can put it on the clojure mailing list.

[talios]

On 7 May 2014, at 6:43, Kevin Livingston wrote:

This seems to happen regardless of clojure-maven-plugin -- the project
I was depending on when I was modulating that was also modulating the
classpath dependency. Sorry about that. Do you think there is
somewhere else I should report this? I guess I can put it on the
clojure mailing list.

You could try the maven users list - I still got your original post and
noticed you've got some TYCHO-SNAPSHOT warning?

c-m-p supports the maven toolchains plugin to control the JVM being used

• not sure if that has any baring.

I did just notice when using 0.1.0 of java.classpath I see your same
behaviour.

About to drive into work so will think about your dependency issue and
reply again if I think of something...

Have you also tried under maven 3.2.1?

[drlivingston]

I test on my machine, but it gets run "for real" on a Hudson-controlled box, and that's using maven 3.1.1 - it had the same difficulties.

I'm glad you are seeing it with 0.1.0 (and confirming I'm not totally crazy) - did you have to have code linked to it, or did just listing dependency cause trouble? I guess we can cook up a minimal example and pass it around.

Don't know anything about what a "tycho" warning would be? I think that's because one of my colleagues added some eclipse .m2 stuff to the C project. But when I was cooking it down I gutted that too and the behavior remained.

thanks for the feedback.
Kevin

[drlivingston]

I know this has sat dormant for a little while but I've realized what I thought was a work-around wasn't.

With clojure-maven-plugin 1.3.23 and java.classpath 0.2.2 I do get more than a single classpath element but I also see the following:

#<File /private/var/folders/bk/w587c61d02s2hkn5zqs4zlnm0000gs/T/clojuremavenplugin7074108075474876645jar>
 "@META-INF/MANIFEST.MF"
 "@sun/net/spi/nameservice/dns/DNSNameService$1.class"
 "@sun/net/spi/nameservice/dns/DNSNameService$2.class"
...

I'm getting some default classes but not most of my code. I'm only seeing sun stuff. no clojure. none of my dependencies. no resource files.

And if I try to force the classloader to open something I know should be there it can't find it either.

I noticed that that first thing ends in "jar" not ".jar" this prevents java.classpath/jar-file? from seeing it as one. But even if I work around that, and expand it's contents I'm not seeing what I should.

[drlivingston]

Comparing my projects to the documentation I notice that my projects are package type jar does that matter?

My configuration also looks like the following, is that acceptable?
(Behavior is the same with and without explicitly listing source directories in configuration.)

      <plugin>
        <groupId>com.theoryinpractise</groupId>
        <artifactId>clojure-maven-plugin</artifactId>
        <!--<version>1.3.20</version>-->
        <version>1.3.23</version>
        <extensions>true</extensions>
        <executions>
          <execution>
            <id>compile-clojure</id>
            <phase>compile</phase>
            <goals>
              <goal>compile</goal>
            </goals>
          </execution>
          <execution>
            <id>test</id>
            <phase>test</phase>
            <goals>
              <goal>test</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <sourceDirectories>
            <sourceDirectory>src/main/clojure</sourceDirectory>
          </sourceDirectories>
          <testSourceDirectories>
            <testSourceDirectory>src/test/clojure</testSourceDirectory>
          </testSourceDirectories>
        </configuration>
      </plugin>

[talios]

On 27 Jun 2014, at 11:59, Kevin Livingston wrote:

Comparing my projects to the documentation I notice that my projects
are package type jar does that matter?

Nope - all the custom package does is allow you to skip the
elements in the configuration.

I'm still not sure whats going on here, I suspect I'll need to dig into
the code for clojure.classpath and see what it's doing - ClassLoaders in
java are tree like, with multiple loaders descending from others - it's
quite possible that's its only showing items from ONE of the class
loaders and confusing us with it's limited output.

Mark

[talios]

Re-reading this. This is standard maven dependency behaviour.

Just having a version of "0.1.0" and means "I would like 0.1.0, but if
anyone else wants something higher - use it".

So when you depend on "0.2.2" in the child, you only get the 0.2.2
dependency.

If you want to keep both versions around, then you'll have to look at
going to OSGi or something.

On 7 May 2014, at 6:43, Kevin Livingston wrote:

Then I have project C that depends on B and classpath 0.2.2. In
this configuration C when running is blinded to the classpath.
Changing the dependency management section in A to use 0.2.2 and C
can see the classpath.
If I gut the source code from B that refers to the classpath but leave
the dependency then everything functions fine in C regardless of what
version I have in A. This is odd because I can't figure out how this
is modulating C's view of the common library. If anything it should
break B by going to the wrong version, right?
If I call mvn:dependency:tree on C I only see version 0.2.2 no
matter what.
In this particular case I happen to have control of A and B so I think
I can upgrade them and not mess anyone up. But this is certainly
weird.

[drlivingston]

I've moved everything up to 0.2.2 for that, so I don't think that's the issue. And I certainly don't need both. (For reference if If I revert everything to clojure-maven-plugin 1.3.9 with classpath 0.2.2, I get the expected/desired behavior.)

classpath reads off clojure.lang.RT/baseLoader and what appears to be all of it's parents.

https://github.com/clojure/java.classpath/blob/0b3401cc97fcd68b7d095312e49ed0952c1a1a04/src/main/clojure/clojure/java/classpath.clj#L63

[drlivingston]

One more data point (sorry for the barrage of comments) the plugin 1.3.9 classpath 0.1.0 mix seems to work everywhere. The plugin 1.3.9 classpath 0.2.2 mix works on two different macs, but that combination has failed on two different linux boxes (oracle jre 1.7 and maven 3.2) -- it's java, I know so how can this be? but it's what I'm seeing. (I'll go make a cross post to classpath, here: http://dev.clojure.org/jira/browse/CLASSPATH-6 )

I can confirm this behavior with all versions of java.classpath on all versions of clojure-maven-plugin 1.3.13 and later.

Starting with version 1.3.13 of clojure-maven-plugin, instead of specifying the Java classpath on the command line with -cp, the plugin generates a temporary JAR file with a manifest file containing the real classpath and launches Java with -jar.

See clojure-maven-plugin pull request #58 containing commit 8d8e90e4.

clojure.java.classpath/classpath looks at the classloader that loaded Clojure, which in this case is a child classloader of sun.misc.Launcher$AppClassLoader. AppClassLoader is a subclass of URLClassLoader but its getURLs method returns only the URL of the JAR file. There is no obvious API for getting the Class-Path property of the manifest file without opening and reading the contents of the JAR file.

[talios]

Maybe we should make the creation of the jar a configuration setting, that when running on windows defaults to true ( where this was intended to solve an issue ) and off for anything else - with an override setting?

[drlivingston]

While that might fix my immediate issues (I'm primarily on mac and linux), this would create weird behavior when porting across platforms, which is kind of against the maven/jvm promise isn't it?

Although, I don't know enough yet to speak definitively on what's "right". The discussion on the clojure.java.classpath parallel issue implies that this is potentially an anti-pattern or distortion of what's "right".