Skip to content

pipe() error and Boost.ASIO not working #2883

Closed
Closed
pipe() error and Boost.ASIO not working#2883
Labels
networkingwontfix

Description

@edubart
edubart
opened on Oct 10, 2014

I'm currently porting an application that does use Boost.ASIO for TCP sockets. Currently I know that emscripten does support TCP networking via websocks and I have to use a websocks proxy for the incoming connections in my server. I have Boost 1.55.0 and latest stable emscripten in my setup. However a simple connect does not work.

Test case:

#include <iostream>
#include <boost/asio.hpp>

void handle_connect(const boost::system::error_code& error)
{
    std::cout << "connected!" << std::endl;
}

int main(int argc, char* argv[])
{
    boost::asio::io_service io_service;
    boost::asio::ip::tcp::endpoint endpoint(boost::asio::ip::address::from_string("127.0.0.1"), 7000);
    boost::asio::ip::tcp::socket socket(io_service);

    socket.async_connect(endpoint, &handle_connect);
    io_service.run();
    return 0;
}
$ em++ main.cpp -o main.js -lboost_system && node main.js
pipe() returning an error as we do not support them

/home/bart/projects/playground/main.js:79
      throw ex;
            ^
5265032

I have a server open in the port 7000 and I was expecting the "connected!" output, but instead got that error. Running in google chrome I get the same error, digging into ASIO source files I have found the root of the problem in boost/asio/detail/impl/pipe_select.interrupter.ipp

void pipe_select_interrupter::open_descriptors()
{
  int pipe_fds[2];
  if (pipe(pipe_fds) == 0)
  ...

So I've created a second test case to see if just that would work:

#include <unistd.h>

int main(int argc, char* argv[])
{
    int fds[2];
    pipe(fds);
    return 0;
}
$ em++ main.cpp -o main.js -lboost_system && node main.js
pipe() returning an error as we do not support them

And I get the same error. And I have a question, is really possible to get Boost ASIO working with emscripten?

Activity

edubart
changed the title [-]Boost.ASIO does not work with emscripten[/-] [+]pipe() error and Boost.ASIO not working[/+] on Oct 10, 2014

kripken commented on Oct 10, 2014

@kripken
kripken
on Oct 10, 2014
Member

We have limited support for networking functions. It should be possible to add more, if they are async. See tests/test_sockets.py for the network tests, which show what definitely works.

juj
added
networking
on Sep 21, 2015

pgarnero06 commented on Mar 11, 2016

@pgarnero06
pgarnero06
on Mar 11, 2016

Hello. I experience the same issue, in Firefox, preventing the usage of boost::asio::local::stream_protocol::socket type. Is there any plan to support pipe() system call ?

cynecx commented on May 22, 2016

@cynecx
cynecx
on May 22, 2016
Contributor

My small 2d game engine uses boost.asio and I wasn't in the mood of implementing my own networking library so I thought fixing this would be a better idea.

I implemented the pipe syscall + did some small changes to boost.asio.

@kripken Do you think we could add that modified asio library to emscripten ports? (It's actually asio without boost, it has the advantage that it is header only so no actual compilation is needed).

I will open a PR for the pipe syscall as soon as I cleaned up the code. :)

kripken commented on May 23, 2016

@kripken
kripken
on May 23, 2016
Member

If it's just a header, maybe ports is overkill, though? Each ports project requires a bunch of setup for compiling etc., which makes sense for larger projects, but maybe not header-only things.

kripken commented on May 23, 2016

@kripken
kripken
on May 23, 2016
Member

Or perhaps we could put it under ports (or somewhere else official) but not have it use the ports infrastructure?

cynecx commented on May 23, 2016

@cynecx
cynecx
on May 23, 2016
Contributor

Yes I agree, also putting this with the other ports has the main advantage that it is much easier to find.

I am currently testing asio and it's working great so far (async_connect/async_write/async_receive/... stuff). But I came across a "race condition" (kinda).

The problem occurs when I send an empty ArrayBuffer (Yes this is possible with WebSockets when using node + ws module as a server). In this special case recvmsg/recv will return zero which indicates that a socket has performed a shutdown (Disconnection/socket -> eof). I think we should have a more robust handling of this special case, for example by ignoring empty "packets" because sending an empty ArrayBuffer shouldn't raise a pseudo disconnect event.

What do you think?

cynecx commented on May 23, 2016

@cynecx
cynecx
on May 23, 2016
Contributor

Pipefs implementation (Tests are missing -> I will open a PR when this is done):
https://github.com/cynecx/emscripten/tree/pipefs
(I am not sure if my bucket allocation algorithm is good enough, performance wise.)

My proposed fix for the empty ArrayBuffer issue:
https://github.com/cynecx/emscripten/tree/sockfs_nullarrbuffer

cynecx commented on May 23, 2016

@cynecx
cynecx
on May 23, 2016
Contributor

Asio patch for emscripten compatibility:
https://github.com/cynecx/asio/tree/emscripten

kripken commented on May 24, 2016

@kripken
kripken
on May 24, 2016
Member

I don't follow, at what level would the more robust handling be?

kripken commented on May 24, 2016

@kripken
kripken
on May 24, 2016
Member

(I opened an asio repo in ports for this.)

cynecx commented on May 24, 2016

@cynecx
cynecx
on May 24, 2016
Contributor

See this inline comment:
cynecx@0b838e6

This "issue" can be triggered by just one line of code when for example using node + websocket module as a server instead of using a server compiled with emscripten ( socket.send(new ArrayBuffer("")) ).

It just doesn't feel right that we can trigger a disconnect event just by sending an empty ArrayBuffer.

pgarnero06 commented on May 24, 2016

@pgarnero06
pgarnero06
on May 24, 2016

Thanks for the fix. I have reported your modifications in my boost.asio sources and installed the emscripten version implementing pipe() => works well. Thanks again.

cynecx commented on May 24, 2016

@cynecx
cynecx
on May 24, 2016
Contributor

@kripken I am currently figuring out how we should deal with thread-safety (Or if we have to). If I understand correctly all syscalls are proxied to the main thread, that means that it is thread safe to call read/write from multiple threads? Should I need to be worried about this?

kripken commented on May 27, 2016

@kripken
kripken
on May 27, 2016
Member

I believe that is correct, yes, we currently proxy all syscalls to the main thread. I don't think there are exceptions to that, but @juj, can you please confirm?

cynecx commented on May 27, 2016

@cynecx
cynecx
on May 27, 2016
Contributor

@kripken I am curious. Wouldn't this significantly reduce the performance of threads? Isn't emscripten taking usage of Atomics (The API addition to SharedArrayBuffer)?

kripken commented on Jun 1, 2016

@kripken
kripken
on Jun 1, 2016
Member

Yes, there are some perf issues with this. However @juj has optimized many things here. Might still be more stuff left to optimize though, in particular filesystem stuff.

atrosinenko commented on Jul 20, 2017

@atrosinenko
atrosinenko
on Jul 20, 2017
Contributor

The @cynecx's pipe implementation was ultimately merged in #4935. You can test it in the incoming version of the SDK.

juj commented on Jul 21, 2017

@juj
juj
on Jul 21, 2017
Collaborator

In MEMFS implementation (src/library_fs.js and src/library_memfs.js etc.), it is thread safe (will not crash) to call read and write and any syscalls from multiple threads, because all filesystem related syscalls are proxied to the main thread.

In the new ASMFS implementation, which implements the filesystem in C/C++ code, similarly read, write and all other filesystem syscalls are expected to be thread safe identical to with MEMFS, although there won't be any proxying needed, which will give much better performance. However the thread safety of ASMFS has not yet been implemented, that is an upcoming TODO item.

stale commented on Aug 30, 2019

@stale
stalebot
on Aug 30, 2019

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.

stale
added
wontfix
on Aug 30, 2019
stale
closed this as completedon Sep 6, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    networkingwontfix

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions