http library

[0dminnimda]

Is there any http library, like pythons 'requests'?
I know it's an unfinished language, just wondering whether anything like that exists, didn't find mentions of such thing in the source code.

[aardappel]

Does not exist yet, sorry.. we've had plans to add one, but likely that should be based on a C++ http library that is small, robust and still featureful, and its hard to find such things. In particular, if you are expecting to support https, there is probably no such thing :)

[shamrin]

I've briefly looked into what Lobster http client can be based on. Observations:

• libcurl with a selection of TLS libraries
• Node.js uses OpenSSL and llhttp with a wrapper.
• Ladybird browser uses its own LibHTTP library. Looks nice, but probably not fully battle-tested yet and hard to decouple from the rest of Ladybird.
• It may be a good idea to use non-C++ library, when the language has great C/C++ interop. For example, Zig, that includes HTTP and TLS in the standard library.
• Python converged on (bring-your-own-IO) h11/h2 + httpcore + httpx. Can't be used in Lobster, but interesting to see what other language communities do.

[aardappel]

Stuff like libcurl is positively gigantic. Making Lobster build 10x as long for http(s) support is not really an option.

Zig is neat but wouldn't want to complicate the build system by involving another compiler. It really needs to be standalone C++ that can be added to the build system of the 6 platforms Lobster builds on.

One option for these possibly gigantic features is to build it as a standalone dll/so, that can optionally be loaded into Lobster. So a lib that combines something like libcurl with some Lobster bindings. Or even keep the bindings in Lobster but just use the dll interface. That way users that don't need http(s) don't pay for it.

[0dminnimda]

I'd say the easiest way would be to precompile a static library and hook it up when building the program. The only requirement is to have ffi in the language.

[shamrin]

Zig can compile to C file, which can later be built without Zig. The .c artifact is unfortunately big (32 Mb) and obviously not for human consumption. I did a quick experiment with Zig http example on my laptop:

$ time zig build-exe main.zig -ofmt=c -OReleaseSmall
        0.99 real         1.10 user         0.09 sys
$ time clang main.c -w -Os -o main -lpthread
       12.65 real        12.45 user         0.15 sys
$ ls -lh
-rwxr-xr-x  1 user  user   725K … main*
-rw-r--r--  1 user  user    32M … main.c
-rw-r--r--  1 user  user   1.3K … main.zig
-rw-r--r--  1 user  user   150K … zig.h

Direct compile, for comparison:

$ time zig build-exe main.zig -OReleaseSmall
        7.49 real         7.02 user         0.21 sys
$ ls -lh main
-rwxr-xr-x  1 user  user   713K … main*

(Removing optimization flags like -Os drops the time down to 4-5 seconds.)

[aardappel]

Looking at this again, I think really what is ideal is a library that is a thin wrapper around whatever http lib the OS already supplies, like Windows has WinHTTP, Apple has NSURLSession etc. That way the lib can stay small and we don't need to include a huge lib in Lobster.

This seems to do exactly that https://github.com/nativeformat/NFHTTP but a) it still compiles the whole of curl/openssl from source on Linux/Android, c) depends on a bunch of other libs (why does http need sqlite?) and c) it seems to largely have been abandoned maintenance wise?

Actually for windows it depends on https://github.com/microsoft/cpprestsdk which also is cross-platform.. why not use that directly? But that then relies on several boost libraries and openssl etc :)

Should be possible on Linux to depend on the system libcurl no?

This is definitely very lightweight, would allow usage of just WinHTTP: https://github.com/microsoft/WinHttpPAL it uses curl thru CMake as find_package(CURL 7.57 REQUIRED MODULE) not sure how easy that is.