Comments (21)
BTW the bug is reported in liburing but response fro authors is "use a decent case sensitive filesystem".
from mediasoup.
This is a bug in a python module used by meson when untar-ing the file. It detects self a pointing link, which is a bug in the module.
I have no idea why it does happen in Docker and not in a non dockered linux.
from mediasoup.
Why does it even try to copy those files in the first place? Those filed should be downloaded inside of container instead.
from mediasoup.
Why does it even try to copy those files in the first place? Those filed should be downloaded inside of container instead.
What do you mean? This happens when running make
or make fuzzer
or make test
(or invoke xxx
) within the Docker container.
from mediasoup.
I mean it might be that worker/out
and worker/subprojects
are copied into the container, but they should not be.
from mediasoup.
See Dockerfile
. It doesn't copy anything but mounts a volume on the mediasoup root folder. The goal is to be able to keep editing original mediasoup source code in your editor while running/testing stuff in Docker.
Said that, it's needed (and I always do) to clean everything in the worker (make clean-subprojects
, make clean-pip
, make clean-all
) once in Docker, otherwise tons of arch issues due to already instaled pip, meson and subprojects. And doing it doesn't avoid this issue.
from mediasoup.
That is what I'm saying, it should probably only mount source code, but not the rest. So instead of mounting the whole root, it should strategically mount only specific things where there are no symlinks. All the downloaded stuff should be inside of the container only.
from mediasoup.
How would you tell Docker to mount this tree?
mediasoup/node/
mediasoup/rust/
mediasoup/worker/src
mediasoup/worker/include
mediasoup/worker/subprojects/ BUT ONLY .wrap files
mediasoup/worker/fuzzer
mediasoup/worker/test
mediasoup/worker/meson...etc
in the same "mediasoup" folder destination without including mediasoup/worker/subprojects
?
from mediasoup.
Anyway I insist:
The scenario of the bug is that we run "make docker" then "make docker-run" to enter the Docker container, then clean absolutely everything in the worker folder including subprojects. Then "make" and it fails. So it wouldn't be fixed by not mounting worker/subprojects folder.
from mediasoup.
The goal of what I was describing is to make sure container works with local (to container) file system, not with paths mounted from the host. If that is achieved, it will not fail anymore.
from mediasoup.
The goal of what I was describing is to make sure container works with local (to container) file system, not with paths mounted from the host. If that is achieved, it will not fail anymore.
Right now the whole mediasoup
folder is mounted in the container. Assuming that we must manually clear worker artifacts once in Docker, what's exactly wrong with that?
And if we do your approach (which involves copying source files to the container) we loose the ability to edit those source files direclty in our editor and work normally. That's a huge drawback I couldn't accept. I run Docker, enter it, run make fuzzer
and run fuzzer stuff. Something fails. I want to edit files in my editor then make fuzzer
and run fuzzer stuff again rather than having to edit with Vi the source files in the Docker container.
from mediasoup.
If this is just for fuzzer, then you probably don't need node
or rust
in there and you can copy *.wrap
files, while mounting worker/fuzzer
, worker/src
and similar strategic directories only.
from mediasoup.
I do run Node as well when in Docker, I don't remember what for but I've done.
from mediasoup.
The only solution I find here is to simply add the MESON_ARGS="-Dms_disable_liburing=true"
env variable to the Dockerfile.
This error comes from the fact that MacOS FS is case insensitive and linux is not. There are ways to make a Docker volume case sensitive in MacOS but I honestly don't think it's worth it.
If we ever need to try liburing in Docker, then we know we need to use a local folder, rather than one from the MacOS host.
from mediasoup.
Is then ok if I write a PR disabling liburing in our Dockerfiles? However we loose the ability to run mediasoup with liburing in Linux Docker in our Macs.
from mediasoup.
However we loose the ability to run mediasoup with liburing in Linux Docker in our Macs.
You can only loose something you have :-)
Edit: and liburing cannot be run in Docker from MacOS right now.
from mediasoup.
Edit: and liburing cannot be run in Docker from MacOS right now.
But it's fetched anyway... and I don't remember why because the Docker Linux kernel in which mediasoup is built is the same than the Docker Linux kernel in which it runs later.
from mediasoup.
I'm fixing this issue in #1338 as a bonus track.
from mediasoup.
As Jose told me, here is the obvious problem in case insensitive file systems: https://github.com/axboe/liburing/tree/master/man
from mediasoup.
Solution in macOS:
- Assuming you have mediasoup in
/Users/xxx/src/mediasoup
. cs $HOME
mkdir src-cs
(cs == case sensitive), or name it as you wish.sudo diskutil apfs addVolume disk1 APFSX src-cs -mountpoint /Users/xxx/src-cs
sudo chown -R $(id -u):$(id -g) src-cs
(ignore the errors).- Now clean worker and pip and
out/
and subprojects in your mediasoup folder. cp -a /Users/xxx/src/mediasoup /Users/xxx/src-cs/mediasoup
- Done.
Now you have mediasoup in a case insensitive file system and make fuzzer
works in Docker Linux.
then change soft links in mediasoup-demo, etc.
from mediasoup.
It's trivial so I'm gonna remove the hack from Dockerfiles.
from mediasoup.
Related Issues (20)
- Multi-thread bug when using usrsctp in N Worker threads in Rust HOT 1
- opus dtx incorrectly judged HOT 7
- Possible missing break in H264_SVC::ParseSingleNalu()
- Must join all XxxConsumer classes into a single one
- set MS_LOG_DEV_LEVEL 3, lead to build errors HOT 1
- #define MS_LOG_DEV_LEVEL 3, build error HOT 1
- SeqManager<T>::SeqLowerThan() crashes when used as compare function in abseil set or map (only in debug mode) HOT 24
- Use uint64_t for containers with RTP seq number as key
- arm64 prebuilt binaries are replacing x64 ones HOT 4
- test-PipeTransport.ts randomly fails HOT 2
- RtpPacket::SetPayloadLength get wrong payloadLength HOT 2
- Possible memory leak HOT 34
- DataConsumer::SendMessage cause onQueuedCallback* cb leak and DATACONSUMER_SEND have no response HOT 1
- FeedbackRtpTransport.cpp may have an overflow mistake HOT 9
- Worker crash when closing `WebRtcServer` if there are TCP connections active HOT 2
- Mediasoup worker died, exiting in 2 seconds... HOT 2
- [Rust] Error log “XXX closing failed on drop: Channel already closed”
- Add interface for RTC::Shared, Channel::ChannelNotifier and ChannelMessageRegistrator HOT 3
- [Rust] Transport with only audio producer results in server-side transport cc is not created. HOT 2
- Bug Simulcast plainTransport -> webrtcTransport
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mediasoup.