Allow not linking against crypto system libraries

[RokasPuzonas]

For my use case I would like to statically link against mbedtls. I would be fine with having an option named "link-system-libraries" which would skip the calls to .linkSystemLibrary(...). The user would be expected the link against the library, I'm fine with libssh2 not pulling in the dependency and requiring the user to have it their own build.zig.zon.

But I'm not sure if this is the best option, I saw that zlib use b.systemIntegrationOption(...), but I haven't found a way for how this option can be provided through b.dependency(...). Because in my application I would like to have 2 modes, first statically linked against mbedtls, seconds dynamically linked against openssl. But from what I have found b.systemIntegrationOption can only be provided through CLI, I would like to change it through b.dependency. Am I missing something? Is there a way to use b.systemIntegrationOption in the way that I want?

What do you think? Which method should be used? The "link-system-libraries" flag or b.systemIntegrationOption()?

[bytesofpie]

@RokasPuzonas Solved in #9 🎉 you can now individually control the link mode for each crypto dependency. Added an example to the README. More detailed output in #9 as well if you need

Let me know if that works for you! Updated the tag commit reference as well, so you should be able to clear cache and re-build

[RokasPuzonas]

@hamptokr I wasn't able to use the mbedtls-linkage option, it seemed that it was still linking dynamically against the library. I'll try to create a minimal reproduction, so I could check if this is something that I am doing wrong or if there actually is a bug.

What OS and zig version did you use? Did you use the master branch of zig? I'm using 0.15.1, it could be that this only works on the master branch.

I'll also create another pull request for fixing sftp support, because currently the src/sftp.c file is not being used from libssh2. Because of this calling any sftp related function seemed to hang execution. If I'm already fixing sftp, I'll also double check if there is nothing else that is missing.

[RokasPuzonas]

Ok I found that if mbedtls is installed on the machine that you are compiling both static and dynamic modes work. But if I uninstall mbedtls, then even compiling in static mode fails. It still tries to search to search for it.

Here is the error message that I get:

Details

error: error: unable to find dynamic system library 'mbedtls' using strategy 'paths_first'. searched paths:
  /usr/local/lib/libmbedtls.so
  /usr/local/lib/libmbedtls.a
  /lib64/libmbedtls.so
  /lib64/libmbedtls.a
  /lib/libmbedtls.so
  /lib/libmbedtls.a
  /usr/lib64/libmbedtls.so
  /usr/lib64/libmbedtls.a
  /usr/lib/libmbedtls.so
  /usr/lib/libmbedtls.a
error: unable to find dynamic system library 'mbedcrypto' using strategy 'paths_first'. searched paths:
  /usr/local/lib/libmbedcrypto.so
  /usr/local/lib/libmbedcrypto.a
  /lib64/libmbedcrypto.so
  /lib64/libmbedcrypto.a
  /lib/libmbedcrypto.so
  /lib/libmbedcrypto.a
  /usr/lib64/libmbedcrypto.so
  /usr/lib64/libmbedcrypto.a
  /usr/lib/libmbedcrypto.so
  /usr/lib/libmbedcrypto.a
error: unable to find dynamic system library 'mbedx509' using strategy 'paths_first'. searched paths:
  /usr/local/lib/libmbedx509.so
  /usr/local/lib/libmbedx509.a
  /lib64/libmbedx509.so
  /lib64/libmbedx509.a
  /lib/libmbedx509.so
  /lib/libmbedx509.a
  /usr/lib64/libmbedx509.so
  /usr/lib64/libmbedx509.a
  /usr/lib/libmbedx509.so
  /usr/lib/libmbedx509.a

Here is how my build.zig looks:

Details

pub fn build(b: *std.Build) void {
    // More code here...

    const root_module = b.createModule(.{
        .root_source_file = b.path("src/main.zig"),
        .target = target,
        .optimize = optimize,
    });

    const mbedtls_dependency = b.dependency("mbedtls", .{
        .target = target,
        .optimize = optimize,
    });

    const libssh2_dependency = b.dependency("libssh2", .{
        .target = target,
        .optimize = optimize,
        .strip = optimize != .Debug,
        .@"crypto-backend" = .mbedtls,
        .@"mbedtls-linkage" = .static,
    });
    const libssh2_artifact = libssh2_dependency.artifact("ssh2");
    libssh2_artifact.linkLibrary(mbedtls_dependency.artifact("mbedtls"));
    root_module.linkLibrary(libssh2_dependency.artifact("ssh2"));

   // More code here...
}

It seems that while compiling the libssh2 compilation unit it requires that the library exists somewhere. It doesn't allow linking against it later. I'll see if any of other options that can be passed to .linkSystemLibrary2() helps. I'll also try checking other zig library to see how they solve this problem, surely somebody else has encountered this.

[bytesofpie]

@RokasPuzonas Ah I see now 💡 Thanks for the investigation! That's expected. We're linking to the system libraries. We vendor zlib so that's why it's not the case there. We could vendor all of the possible dependencies, but we'd need to upgrade the other wrapped libraries to zig 0.15 too. For example, if I try to vendor https://github.com/allyourcodebase/mbedtls it fails as it's build.zig is incompatible with 0.15

[RokasPuzonas]

Even though I didn't want to use b.systemIntegrationOption, it seems that this option is exactly designed to solve this use case ziglang/zig#18778.

For now I think it's best to wrap the .linkSystemLibrary2() calls behind a if (b.systemIntegrationOption()) and by default make it dynamically linked. For the statically linked versions I think it would be fine to leave a // TODO: comment and leave linking against the static library to the users responsibility. I'll create a PR.

Ya, I noticed that mbedtls hasn't been updated. For my application I'm temporarily fetching this dependency https://github.com/dasimmet/mbedtls.git, it's an updated version to support 0.15.1. But I would like to avoid using this, in the main branch for libssh2. I don't like the idea of pushing this fork on the users.