Main API tries and fails to load an optional dependency

[jvasileff]

Running a basic hello-world program with:

module com.example.helloworld "1.0.0" {
    import "com.google.guava:guava" "18.0";
}

using:

java -cp $(ceylon classpath --rep aether com.example.helloworld/1.0.0) \
    com.redhat.ceylon.compiler.java.runtime.Main \
    com.example.helloworld/1.0.0 \
    com.example.helloworld.run_

fails with:

Exception in thread "main" java.lang.RuntimeException: com.redhat.ceylon.compiler.java.runtime.Main$ClassPath$ModuleNotFoundException: Module com.google.code.findbugs:jsr305/1.3.9 not found
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:546)
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:553)
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:553)
    at com.redhat.ceylon.compiler.java.runtime.Main.setupMetamodel(Main.java:528)
    at com.redhat.ceylon.compiler.java.runtime.Main.runModule(Main.java:474)
    at com.redhat.ceylon.compiler.java.runtime.Main.main(Main.java:580)
Caused by: com.redhat.ceylon.compiler.java.runtime.Main$ClassPath$ModuleNotFoundException: Module com.google.code.findbugs:jsr305/1.3.9 not found
    at com.redhat.ceylon.compiler.java.runtime.Main$ClassPath.loadModule(Main.java:297)
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:542)
    ... 5 more

The jsr305 module is an optional dependency of Guava - see https://repo1.maven.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom

ceylon classpath ... lists guava-18.0.jar, but, as expected, not jsr305.jar.

One interesting item: I have no idea where Main is getting the version number in jsr305/1.3.9. Guava's pom.xml does not specify a version.

[quintesse]

@jvasileff See https://repo1.maven.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom for the JSR version.

[quintesse]

Btw @FroMage , as you can see it's not during compilation but at runtime that this error occurs. It seems it's trying to register the metamodel for the optional module which isn't available. At least that's my guess.

[quintesse]

NB: should the ceylon classpath tool have an --include-optionals flag or something? Or perhaps the default should be to include them (better safe than sorry) and have an --exclude-optionals?

[jvasileff]

Oh, that makes sense. And, it's not listed as optional in the parent pom; maybe that has something to do with it.

On a related note, it looks like Main makes a network connection to download poms if necessary. I guess that makes sense, especially here, but it's not something I'd want on a production server.

[quintesse]

but it's not something I'd want on a production server.

Try setting the offline mode. (I'm not really sure if the Main API makes that easy to do, @FroMage ?)

[FroMage]

In theory the Main API will not do network lookups, but that's not true for Maven, because we haven't found the flag in aether to make it not do remote lookups.

As for including the optional jars by default in classpath, I guess that's a good idea, but would it fix the problem here?

[quintesse]

because we haven't found the flag in aether to make it not do remote lookups.

That was already implemented some time ago, it should work.

[quintesse]

(the thing we don't support yet are timeouts because the Aether stuff doesn't have an API for that)

[FroMage]

Not for this IIRC.

[FroMage]

See #87 (comment)

[quintesse]

Are you sure? I remember this was tested when we added it to the resolver:

ceylon-module-resolver/maven/src/main/java/com/redhat/ceylon/cmr/maven/AetherUtils.java

Line 318 in 156f46f

ConfigurableMavenResolverSystem factory = Resolvers.use(ConfigurableMavenResolverSystem.class, classLoader).workOffline(offline);

Wasn't the problem with that issue that you can't just read the information from the POM without it trying to download the artifacts (when offline=false)

[FroMage]

Yes, as the comments say, there's no way to make aether read just the pom file without resolving. I suppose that if I set it to offline it will not hit the net, though, so it should stick to the local cache, which should work.

[alesj]

I've added this test and it passes:

@Test
public void testOptionalDependency() throws Throwable {
    Repository repository = AetherRepository.createRepository(log, false, 60000);
    RepositoryManager manager = new SimpleRepositoryManager(repository, log);
    ArtifactResult result = manager.getArtifactResult("com.google.guava:guava", "18.0");
    Assert.assertNotNull(result);
    Assert.assertEquals(result.name(), "com.google.guava:guava");
    File artifact = result.artifact();
    boolean exists = false;
    try {
        Assert.assertNotNull(artifact);
        Assert.assertTrue(artifact.exists());
        exists = true;
        Assert.assertTrue(result.dependencies() == null || result.dependencies().isEmpty());
    } finally {
        if (exists) {
            Assert.assertTrue(artifact.delete()); // delete this one
        }
    }
}

[quintesse]

@alesj This wasn't so much about the CMR not finding the artifacts but about AetherUtils returning the wrong information about them.

@jvasileff This seems to work now for me. Could you confirm so we can close this?

[jvasileff]

@quintesse great, that helps! Indeed, java -cp ... Main ... works for me now.

But, I'm afraid there is more to this. The following still fail:

Running without the Main api:

$ ceylonDev run simple --rep aether
ceylon run: Could not find module: com.google.guava/guava/18.0 (invalid
version?)

Running without the Main api, without guava, but with import "com.google.code.findbugs:jsr305" "1.3.9";

$ ceylonDev run simple --rep aether
ceylon run: Could not find module: com.google.code.findbugs/jsr305/1.3.9
(invalid version?)

Running with the Main api, with or without guava, but with import "com.google.code.findbugs:jsr305" "1.3.9";

$ java -cp $(ceylonDev classpath --rep aether simple/1.0.0) com.redhat.ceylon.compiler.java.runtime.Main simple/1.0.0 simple.run_
Exception in thread "main" java.lang.RuntimeException: com.redhat.ceylon.compiler.java.runtime.Main$ClassPath$ModuleNotFoundException: Module com.google.code.findbugs:jsr305/1.3.9 not found
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:556)
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:566)
    at com.redhat.ceylon.compiler.java.runtime.Main.setupMetamodel(Main.java:538)
    at com.redhat.ceylon.compiler.java.runtime.Main.runModule(Main.java:484)
    at com.redhat.ceylon.compiler.java.runtime.Main.main(Main.java:593)
Caused by: com.redhat.ceylon.compiler.java.runtime.Main$ClassPath$ModuleNotFoundException: Module com.google.code.findbugs:jsr305/1.3.9 not found
    at com.redhat.ceylon.compiler.java.runtime.Main$ClassPath.loadModule(Main.java:304)
    at com.redhat.ceylon.compiler.java.runtime.Main.registerInMetamodel(Main.java:552)
    ... 4 more

Note: ceylonDev classpath --rep aether simple/1.0.0 does include the findbugs jar when imported in module.ceylon.

So, I suspect there are two issues:

• ceylon run appears to still want to load the optional dependency
• The module loader cannot make sense of the findbugs jar, since it is missing pom.xml. (updated - fixed grep cmd)

$ unzip -l /Users/jvasileff/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar | grep pom
$

For reference, my module.ceylon file:

module simple "1.0.0" {
  // comment/uncomment one or both of these: 
  import "com.google.guava:guava" "18.0";
  import "com.google.code.findbugs:jsr305" "1.3.9";
}

[quintesse]

@jvasileff you're using a wrong command line syntax. Everything after the module name for "ceylon run" are supposed to be arguments for the program you're running. So the --rep aether has to come before the module name. (But after the word "run")
(Edit: fixed mistake in text)

[jvasileff]

Oops.

So ceylon run looks like it works in all cases. The only remaining issue seems to be the Main API w/findbugs imported, which I'm guessing would also affect the Tool Provider API, although I haven't tested that, yet.

[quintesse]

I tried it with both imported and it still seemed to work fine. What problem are you seeing?

[quintesse]

Ah no, I have reproduced it

[jvasileff]

Not a super controlled test, but it looks like the tool provider API is indeed affected:

   [tomcat] Caused by: java.lang.RuntimeException: Missing module: com.google.code.findbugs:jsr305/1.3.9
   [tomcat]     at com.redhat.ceylon.compiler.java.runtime.tools.impl.JavaRunnerImpl.loadModule(JavaRunnerImpl.java:212)
   [tomcat]     at com.redhat.ceylon.compiler.java.runtime.tools.impl.JavaRunnerImpl.loadModule(JavaRunnerImpl.java:224)
   [tomcat]     at com.redhat.ceylon.compiler.java.runtime.tools.impl.JavaRunnerImpl.<init>(JavaRunnerImpl.java:65)
   [tomcat]     at com.redhat.ceylon.compiler.java.runtime.tools.CeylonToolProvider.getRunner(CeylonToolProvider.java:27)
   [tomcat]     at com.redhat.ceylon.war.WarInitializer.initialize(WarInitializer.java:85)

[quintesse]

Well it's indeed because the findbugs jar does not have any meta data at all.

[jvasileff]

Incidentally, compiling with jsr305 fails if you also need import javax.annotation "7";. jsr305.jar includes some javax.annotation classes, but not the one that causes the compile error in my case, which is javax.annotation.PostConstruct.

source/.../Repository.ceylon:13: error: imported declaration not found: 'postConstruct'
    postConstruct
   ^
source/.../Repository.ceylon:38: error: function or value does not exist: 'postConstruct'
    postConstruct shared
   ^

[quintesse]

Well to be honest @jvasileff I'd like to close this issue if you don't mind because I think the original problem is now fixed. Please open a new issue in ceylon.language for this new problem with dependencies that have no (useful) meta-data.

[jvasileff]

@quintesse ok, done. Thanks.