Activity

1. bayes_boundary (lord-fourth0107)

   added

   bug

   on Dec 2, 2021
2. github-actions

   added

   IoTSDK

   on Dec 2, 2021
3. Tim Taylor (timtay-microsoft)

   added

   enhancement

   and removed

   bug

   on Dec 2, 2021
4. timtay-microsoft commented on Dec 2, 2021

   

   Tim Taylor (timtay-microsoft)

   on Dec 2, 2021

   Member

   More actions

   Marking this as "Enhancement" since you are asking for us to support GraalVM
5. jamdavi

   self-assigned this

   on Dec 6, 2021
6. jamdavi commented on Dec 16, 2021

   

   jamdavi

   on Dec 16, 2021

   Contributor

   More actions

   This is due to usage of SecureRandom in the DigestProxyChallengeProcessorImpl class up the chain. Change your Graalvm command line specified in your application.properties to this and it will build just fine. Notice I removed SecureRandom from the initialize-at-runtime command line.

   If you experience issues with azure identity after the build you'll need to work with that team to understand why it's failing.

   quarkus.native.additional-build-args=--initialize-at-run-time=com.microsoft.azure.proton.transport.proxy.impl.DigestProxyChallengeProcessorImpl\\,com.microsoft.aad.msal4jextensions.persistence.linux.ISecurityLibrary\\,com.microsoft.aadmsal4jextensions.persistence.linux.ISecurityLibrary\\,com.microsoft.aad.msal4jextensions.persistence.mac.ISecurityLibrary\\,com.sun.jna.platform.win32.Crypt32\\,com.sun.jna.platform.win32.Kernel32\\,com.nimbusds.jwt.JWTClaimsSet,--allow-incomplete-classpath
   
7. jamdavi

   added

   Underlying Library IssueIssue is a result of a known issue with a 3rd party library that we useIssue is a result of a known issue with a 3rd party library that we use

   and removed

   enhancement

   on Dec 16, 2021
8. lord-fourth0107 commented on Dec 16, 2021

   

   bayes_boundary (lord-fourth0107)

   on Dec 16, 2021

   Last edited by lord-fourth0107

   Author

   More actions

   Hi jamdavi. Thank You for the reply . I tried to build the native image using the changes you suggested and the build was success. But when deploying the native image i now get some errors from Azure Identity dependency . This is basically due to non-availability of http client . When adding the http client dependency , the build fails. Similar but not exactly same error was found when i tried for jvm build . Let me know in case of any updates
9. jamdavi commented on Dec 17, 2021

   

   jamdavi

   on Dec 17, 2021

   Contributor

   More actions

   @uttamsingh0107 I believe you have an internal ticket open on this and I'll respond with this same information there. I'll also close this issue based on my findings below. But this will be helpful for anyone else who needs this information.

   I was able to get this to build with the following changes.

   1. I removed the azure-core-http-netty exclusion under the azure-identity dependency in the POM

   2. I changed the quarkus.native.additional-build-args parameters to the following:

   
   quarkus.native.additional-build-args=--initialize-at-run-time=reactor.netty.tcp.TcpClientSecure\\,reactor.netty.tcp.SslProvider\\,reactor.netty.http.client.HttpClientSecure\\,reactor.netty.resources.DefaultLoopKQueue\\,reactor.netty.resources.DefaultLoopEpoll\\,reactor.netty.resources.DefaultLoopNativeDetector\\,com.microsoft.azure.proton.transport.proxy.impl.DigestProxyChallengeProcessorImpl\\,com.microsoft.aad.msal4jextensions.persistence.linux.ISecurityLibrary\\,com.microsoft.aadmsal4jextensions.persistence.linux.ISecurityLibrary\\,com.microsoft.aad.msal4jextensions.persistence.mac.ISecurityLibrary\\,com.sun.jna.platform.win32.Crypt32\\,com.sun.jna.platform.win32.Kernel32\\,com.nimbusds.jwt.JWTClaimsSet,--allow-incomplete-classpath,--enable-https
   
   

   So, the question is, Is the build failing because there's a problem with my app or a dependency? And, when we do get it working it will fail when you run the app? Is this a problem with the IoT SDK? The Azure SDK? Netty?

   Short answer

   No.

   The Long Answer

   Since GraalVM (GVM) does ahead-of-time compilation, there are bound to be limitations and concerns about the system it's compiled on and the one it's compiled for. A very obvious example would be if I did a local native compile on Windows it won't work for Linux. Similarly, but more nuanced is, if I built on Linux with a 2.0 kernel it likely won't work on a 5.0 kernel. Just as with platform implementation, the same applies for CPU architectures and endianness.

   GVM has a a large set of rules that it looks out for. You've been hit by two of them which are documented:

   1. SecureRandom

   2. Initialization at build time

   SecureRandom

   Since SecureRandom is meant to be read at the instant it's needed GVM will search for any instances of it in memory at build time and block on usage. It does this because if it captured state of the RNG entropy at build time it would be locked in forever and you'd have a static seed for all randoms. This is bad.

   From the docs:

   The SecureRandom implementations open the /dev/random and /dev/urandom files which are used as sources for entropy. These files are usually opened in class initializers. To avoid capturing state from the machine that runs the Native Image builder these classes need to be initialized at run time.

   GVM's requirement is that "these classes need to be initialized at run time". So you need to trace back and find what class(s) instantiate(s) SecureRandom. Any library that uses SSL (or any RNG) will have this requirement.

   Initialization at build time

   If you notice in the build parameters above all but three Microsoft owned classes are specified. For all classes listed they are specified because they initialize SecureRandom in some way, but others are specified because they potentially initialize other classes that are not considered safe, or are unknown to GraalVM.

   This is another requirement of ahead-of-time compilation and not due to a specific library implementation.

   Conclusion

   The exercises I did to get the build working required no changes to your application code, nor the Microsoft SDKs. But instead, required me to follow the GVM instructions and do some iterative changes to the build parameters to satisfy the GraalVM build. The same will be true for runtime exceptions, if libraries (like, azure-core-http-netty) are not included things will fail. If an library has platform dependent native hooks they might also fail.

   The amount and depth of errors that arise from dependent classes we use is all due to how GVM handles ahead-of-time compilation and not because our library doesn't officially support GVM.

   Officially, we haven't tested with GraalVM. There are likely scenarios and edge cases that won't work without some deep investgation. But, the exercises performed while investigating this ticket state that we likely do handle the majority of cases when GVM is setup correctly. The ground work would need to be done by the user of GVM to get the application built and run. Once we start seeing callstacks that point to our SDK not handling a specific scenario we can start to narrow down the issues.

   For now the problems are not specific to our SDK and need GraalVM experts to get the application stood up.
10. jamdavi

    closed this as completedon Dec 17, 2021
11. JayGhiya commented on Jan 3, 2022

    

    Jay Ghiya (JayGhiya)

    on Jan 3, 2022

    More actions

    jamdavi -> Thankyou for detailed answer. Couple of comments- this issue was created to get a graalvm compatible sdk. It is well known that graalvm does build time class initialization as you mentioned which is not a limitation but a trade off that developer decides for very low startup times(8x-10x lower time than jvm) and lower processing times. Thats what high performance and low footprint langs like golang and rust will do. So it is not definitely a limitation but a choice according to app requirements.

    Now coming to making secure random work with graalvm requires changes and recommendations from graalvm experts. This was the intent of creating the github issue in the first place. Also -> there have been multiple libs that already use secure random and have been modified to work with graalvm. i would want to reopen the ticket.
12. jamdavi

    changed the title [-]GraalVM Native build fails when using azure-identity pom dependencies [/-] [+]GraalVM Native build needs runtime initialization command line parameters[/+] on Jan 5, 2022
13. jamdavi commented on Jan 5, 2022

    

    jamdavi

    on Jan 5, 2022

    Contributor

    More actions

    All instances of SecureRandom in our library are instantiated at runtime. It's the transitive dependencies that are the issue here.

    It appears that we can get a build with the existing SDK using well supported and documented methods.

    I will move this issue to a discussion as it's a large feature request and might need to be part of our v2 efforts.
14. Azure

    locked and limited conversation to collaborators on Jan 5, 2022
15. jamdavi

    converted this issue into a discussion #1449 on Jan 5, 2022