chenxinfeng4 / ffmpegcv Goto Github PK
View Code? Open in Web Editor NEWThe ffmpegcv is a ffmpeg backbone for open-cv like Video Reader and Writer
The ffmpegcv is a ffmpeg backbone for open-cv like Video Reader and Writer
my env
cuda 11.3
cudnn 8.2.1
pip install tensorflow-gpu==2.7.0
pip install torch==1.12.0+cu113 torchaudio==0.12.0+cu113 torchvision==0.13.0+cu113 --extra-index-url https://download.pytorch.org/whl/cu113
opencv-contrib-python 4.5.3.56
ffmpeg version N-87353-g183fd30 Copyright (c) 2000-2017 the FFmpeg developers
libavutil 55. 76.100 / 55. 76.100
libavcodec 57.106.101 / 57.106.101
libavformat 57. 82.101 / 57. 82.101
libavdevice 57. 8.101 / 57. 8.101
libavfilter 6.105.100 / 6.105.100
libswscale 4. 7.103 / 4. 7.103
libswresample 2. 8.100 / 2. 8.100
libpostproc 54. 6.100 / 54. 6.100
when i change to ffmpegcv.VideoCapture it read all frame. i think there is a problem with cuda config
first of all, great job!, I wanted to ask if it is possible to add the option to get frames from an rtsp route, something like
'rtsp://admin:admin@ip_cam/'
Hello ! I was using ffmpegcv in windows, and it worked perfectly fine. But when I shifted to Ubuntu22 and installed it could not read stream from same cameras. ffmpegcv is installed without any error. But When I tried to read rtsp stream it is not reading any frame. Although opencv works fine, but I want to use ffmpegcv instead of opencv. Please help me in this regard.
Great work! Do u plan to support reading frames by indexes (random access) ?
Does "ffmpegcv.VideoCaptureStream" support GPU decoding?
How to reduce the buffer of VideoCaptureStream as little as possible?
For some purpose, when the VideoCaptureStream starts receiving the frame, I want it to send immediately that frame to the data process instead of buffering.
How to achieve that with ffmpegcv? And is there a way to do it with OpenCV?
How to set compression rate (crf in case of ffmpeg) when running h264 video writer ?
If I have a self-complied ffmpeg library which support other hardware acceleration such as Rockchip (which is quite popular now), can I compile your library on the top of this ffmpeg lib?
Hello,
Python newbie here, i want to know what i have to pass in cuda here
recommand when using cuda
pip install ffmpegcv[cuda]
is it my nvidia driver version, toolkit version or cudnn path, i am on windows 11.
Hello, thank you so much for your contribution.
I was running the following code shown in the documentation:
from ffmpegcv.ffmpeg_reader_camera import query_camera_options
options = query_camera_options(0) # or query_camera_options("Integrated Camera")
print(options)
i got AssertionError on line 59 in _query_camera_divices_win:
assert len(matches) == len(alternative_names)
I modified line 55 in order to get the first matches. It was a regex problem:
from this:
pattern = re.compile(r'\[[^\]]*?\] "([^"]*)"')
to this:
pattern = re.compile(r'"([^"]*)" ')
i ran the code again and now it works as expected.
Is it possible if you modify this expression in your source code?
line 55 in ffmpeg_reader_camera.py
Thanks in advance
Bruno
Hello, excuse me if this is a very basic question, I am still learning.
I tried an approach that was similar to this:
import ffmpegcv
vfile_in = 'A.mp4'
vfile_out = 'A_h264.mp4'
vidin = ffmpegcv.VideoCapture(vfile_in)
vidout = ffmpegcv.VideoWriter(vfile_out)
with vidin, vidout:
for frame in vidin:
vidout.write(frame)
but instead of vidout = ffmpegcv.VideoWriter(vfile_out)
I wanted to have this:
vidout = ffmpegcv.VideoWriter(os.path.join(output_temp, 'frame_' + str(i).zfill(6) + "." + image_extension))
Sadly the output, whilst having the correct name it didn't maintain or parse the image data ( it didn't have any height, width and it even showed up as as corrupted image in Windows ).
Whilst I am here, I'd like to know if there's a way, similar to my current FFMPEG approach to speeds things up
ffmpeg_command = f'{ffmpeg_path} -i "{video_file}" -vsync 0 -qmin 1 -qscale:v 1 -v quiet -stats "{output_temp}"/"{frame_name}"."{image_extension}'
Particularly, adding " -vsync 0 -qmin 1 -qscale:v 1 " have virtually doubled the performance, granted it has a higher overall size but space isn't an issue, speed is.
Could this be replicated, and if so how?
And another one, could ffmpegcv support Asynchronous encoding, as in encoding frame by frame so that each time I process a frame ( extract the frames, upscale them and then encode ) I could also have it be encoded without having to wait for all of the images to be upscaled and only then start the encoding. Basically encoding on the go, I guess?
cap=ffmpegcv.VideoCaptureStream(url)
while cap.isOpened():
//运行一段时间后isOpened()会返回False
pass
cap=ffmpegcv.VideoCaptureStream(url)
shape = cap.out_numpy_shape
// cap.release() //这时是正常的
Process(target="child_prc", args=(url, shape))
// cap.release() // 这时会直接卡住
When i use hevc
as videowrite encoder and not set preset
,i get error as pic
from this error, i track the code and find the default preset = p2
in ffmpeg_writer.py
I use ffmpeg -h encoder=nvenc_hevc
to check preset
,but 'p2' is not in options
In order to solve this problem, either force the preset
parameter to be set, or modify the default value setting here in the source code.
objective
can use cv2.VideoCapture() function with source file
steps
!wget -c https://cdn.creatomate.com/demo/mountains.mp4
pip install -U opencv-python ffmpegcv
code
import cv2
import ffmpegcv
input_file = 'mountains.mp4'
output_file = 'flipud.mp4'
stream = cv2.VideoCapture(input_file)
vidout = ffmpegcv.VideoWriter(output_file , 'hevc', pix_fmt = 'bgr24')
while True:
(grabbed, frame) = stream.read()
vidout.write(frame)
stream.release()
vidout.release()
result
AttributeError Traceback (most recent call last)
[<ipython-input-13-5428b4c0b500>](https://localhost:8080/#) in <cell line: 7>()
7 while True:
8 (grabbed, frame) = stream.read()
----> 9 vidout.write(frame)
10 stream.release()
11 vidout.release()
[/usr/local/lib/python3.10/dist-packages/ffmpegcv/ffmpeg_writer.py](https://localhost:8080/#) in write(self, img)
65 height = int(height_15 / 1.5)
66 else:
---> 67 height, width = img.shape[:2]
68 self.width, self.height = width, height
69 self.in_numpy_shape = img.shape
AttributeError: 'NoneType' object has no attribute 'shape'
best regards
This causes race condition, when video file is still being written by running ffmpeg process while FFmpegWriter
object is released.
Details:
run_async()
in video_info.py
actually returns the PID of the shell, not the PID of ffmpeg process.
So release_process()
waits for shell process completion, not ffmpeg process completion.
On some environments (like in Google Colab) it takes quite a while for ffmpeg to finish.
The following code demonstrates the problem:
import numpy as np, ffmpegcv
import psutil
img = np.zeros((100, 100, 3))
fn = 'test.mp4'
w, h = 100, 100
writer = ffmpegcv.VideoWriter(fn, None, 30, (w, h))
for i in range(1000):
if i % 10 == 0:
img = np.random.randint(0,255,(w,h), dtype='uint8')
writer.write(img)
for p in psutil.process_iter(['name']):
print(p)
print(">>>>>", writer.process.pid)
writer.release()
for p in psutil.process_iter(['name']):
print(p)
The sample output is like this:
psutil.Process(pid=25371, name='sh', status='sleeping', started='02:05:35')
psutil.Process(pid=25372, name='ffmpeg', status='sleeping', started='02:05:35')
psutil.Process(pid=25383, name='sleep', status='sleeping', started='02:05:36')
>>>>> 25371
As you can see, the writer.process.pid == 25371, which corresponds to pid=25371, name='sh'
, not ffmpeg.
nice to support async ?
the ffmpeg parameter can be run using another python ffmpeg module
e.g. module
typed-ffmpeg
ffmpeg-python
e.g. code
import ffmpeg
url = 'http://takemotopiano.aa1.netvolante.jp:8190/nphMotionJpeg?Resolution=320x240&Quality=Standard&Framerate=30'
output = 'rtsp_%Y_%m_%d_%H_%M_%S.mp4'
scale = 'scale = 320 : -1'
segment_time = '00:01:00'
stream = ffmpeg.input(url).output(filename = output,
vcodec = 'libx264', #acodec = 'copy',
reset_timestamps = 1,
strftime = 1,
f = 'segment',
segment_time = segment_time,
segment_atclocktime = 1,
vf = scale,
r = 25, ).overwrite_output().run_async()
to stop async
stream.terminate()
best regards
I have configured ffmpeg with GPU Support on Google Colab GPU Instance using the Nvidia Official Docs but despite this I was unable to read a video utilizing GPU method ffmpegcv.VideoCaptureNV() .Also the ffmpeg build with CUDA support is successful and verified.
I have made my Google Colab Notebook public where all of the steps are implemented. Could you please have a look at it and advise me on a possible way to circumvent this issue.
I saw the VideoCaptureCAM() function, passing "/dev/video0" to camname doesn't seem to work.
Is there any usage with the camera?
Hi,
Everything works, except when I try to write a video using a different dimension from the input.
I'm reading a movie with vidin = ffmpegcv.VideoCapture(vfile_in)
and vfile_in has a (1680, 1050) size. Then I'm adding some margins to the frame so I can crop it (some points eventually can lie outside dimensions) both using opencv, and writing it with vidout = ffmpegcv.VideoWriter(vfile_out, None, vidin.fps, frameSize=(54, 54))
.
It returns no error but produces a really small .mp4 file that can't be read.
If I just plot the images using opencv, without writing to file, they are fine and with the right dimensions (54, 54).
What am I doing wrong here? Thanks!!
'ffprobe -v quiet -print_format xml -select_streams v:0 -show_format -show_streams "rtsp://ipcam:8554/main"' returned non-zero exit status 1.
can you help me?
Hey,
is there a way how to index certain frame range with ffmpegcv? Something like this (in opencv):
for fra in frames:
capture.set(cv2.CAP_PROP_POS_FRAMES, fra) ##THIS SPECIFICALLY
ret, frame = capture.read()
if ret:
#cv2.imshow('frame',frame)
#cv2.waitKey(1000)
out.write(frame)
# display th loading image with im.show() to see if the frames are loaded correctly
if not ret:
# print('a frame was dropped: ' + str(fra))
capture.release()
# out.release()
Thank you!
nv是可以对rtsp硬解
Is there an equivalent of cv2.VideoWriter's .isOpened()?
How to tell that a videowriter file handle is open or not?
Hello, I've just installed the package and was trying to do a simple camera capture to try it out. So I ran the following code and got the error bellow.
import ffmpegcv
cap = ffmpegcv.VideoCaptureCAM(0)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\__init__.py", line 330, in VideoCaptureCAM
return FFmpegReaderCAM.VideoReader(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\ffmpeg_reader_camera.py", line 277, in VideoReader
id_device_map = query_camera_devices()
^^^^^^^^^^^^^^^^^^^^^^
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\ffmpeg_reader_camera.py", line 94, in query_camera_devices
result = {
^
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\ffmpeg_reader_camera.py", line 59, in _query_camera_divices_win
assert len(matches) == len(alternative_names)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
To further explore the error I ran:
from ffmpegcv.ffmpeg_reader_camera import query_camera_devices
devices = query_camera_devices()
And got the error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\ffmpeg_reader_camera.py", line 94, in query_camera_devices
result = {
^
File "C:\dev\code\faze-exercise\venv\Lib\site-packages\ffmpegcv\ffmpeg_reader_camera.py", line 59, in _query_camera_divices_win
assert len(matches) == len(alternative_names)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
I am running an environment with Python 3.11.2, and ffmpegcv is version 0.3.3. I got ffmpeg version 6.0.
objective
read rtsp and write using ffmpegcv
steps
pip install -U rtsp ffmpegcv
code
import rtsp
import ffmpegcv
url = 'http://clausenrc5.viewnetcam.com:50003/nphMotionJpeg?Resolution=320x240' # aspect ratio = 4:3
client = rtsp.Client(rtsp_server_uri = url, verbose = False)
output_file = 'rtsp_ffmpegcv_time.mp4'
vidout = ffmpegcv.VideoWriter(output_file, None, pix_fmt = 'bgr24')
i = 0
while i <= 9:
frame = client.read(raw = True)
vidout.write(frame)
i += 1
client.close()
vidout.release()
result
AttributeError Traceback (most recent call last)
[<ipython-input-29-fc5430a80232>](https://localhost:8080/#) in <cell line: 11>()
12 frame = client.read(raw = True)
13
---> 14 vidout.write(frame)
15
16 i += 1
[/usr/local/lib/python3.10/dist-packages/ffmpegcv/ffmpeg_writer.py](https://localhost:8080/#) in write(self, img)
65 height = int(height_15 / 1.5)
66 else:
---> 67 height, width = img.shape[:2]
68 self.width, self.height = width, height
69 self.in_numpy_shape = img.shape
AttributeError: 'NoneType' object has no attribute 'shape'
best regards
Hi ffmpegCV Team,
I ran the code for video read & write with ffmpegcv ( CUDA acceleration ) on Google Colab GPU Instances .The Code runs without error on Tesla T4 Instance but gives a BrokenPipeError when the same code is run on Nvidia A100.Could you please help me out in understanding why such a thing would occur
PFA the link for the public Colab Notebook that contains the code alongwith the error message: ffmpegcv notebook
Given Below is the Code and Error that I received.
import ffmpegcv
from tqdm.notebook import tqdm
vfile_in = '/content/youtube_traffic_videos/traffic_video_1.mp4'
vfile_out = "/content/traffic_video_output_gpu_A100.avi"
cap_gpu = ffmpegcv.VideoCaptureNV(vfile_in,gpu=1) #NVIDIA GPU0
out_gpu = ffmpegcv.VideoWriterNV(vfile_out,"hevc", cap_gpu.fps,gpu=1)
nframe = len(cap_gpu)
t = tqdm(total=nframe,desc="Reading + Writing Video-File using A100 GPU",leave=True)
while True:
ret, frame = cap_gpu.read()
if ret:
out_gpu.write(frame)
t.update(1)
if not ret:
break
pass
cap_gpu.release()
out_gpu.release()
---------------------------------------------------------------------------
BrokenPipeError Traceback (most recent call last)
<ipython-input-25-a5ed10679eab> in <module>
14 ret, frame = cap_gpu.read()
15 if ret:
---> 16 out_gpu.write(frame)
17 t.update(1)
18 if not ret:
/usr/local/lib/python3.9/dist-packages/ffmpegcv/ffmpeg_writer.py in write(self, img)
57 assert self.size == (img.shape[1], img.shape[0])
58 img = img.astype(np.uint8).tobytes()
---> 59 self.process.stdin.write(img)
60
61 def release(self):
BrokenPipeError: [Errno 32] Broken pipe
HI
when I use VideoCapture with crop and gray pixel format the beginning for each frames gets added to the end of the last frame so the frames gets exported as show in this image.
here is the code that exported these frames
from ffmpegcv import VideoCapture
import cv2
cap = VideoCapture('test2.mp4', pix_fmt='gray',
crop_xywh=(332, 57, 84, 35))
frameCounter = 0
while frameCounter < ((23*60+14)-(2*60+23)) * 60:
frameCounter += 1
try:
ret, frame = cap.read()
if not ret:
break
except:
break
if frameCounter % 60 != 0:
continue
cv2.imwrite('out/frame'+str(frameCounter)+'.jpg', frame)
frameCounter += 1
pass
cap.release()
cap.release()
code
input_file = 'http://195.196.36.242/mjpg/video.mjpg'
output_file = 'rtsp.mp4'
vidin = ffmpegcv.VideoCapture(input_file, pix_fmt = 'bgr24') # bgr24 (default), rgb24, gray
vidout = ffmpegcv.VideoWriter(output_file, 'hevc', vidin.fps, pix_fmt = 'bgr24')
i = 0
with vidin, vidout:
for frame in vidin:
vidout.write(frame)
if i == 9:
break
i += 1
vidout.release()
vidin.release()
result
AssertionError Traceback (most recent call last)
[<ipython-input-4-e4d2397485bb>](https://localhost:8080/#) in <cell line: 4>()
2 output_file = 'rtsp.mp4'
3
----> 4 vidin = ffmpegcv.VideoCapture(input_file, pix_fmt = 'bgr24') # bgr24 (default), rgb24, gray
5
6 # codec available value None, 'h264', 'hevc'
1 frames
[/usr/local/lib/python3.10/dist-packages/ffmpegcv/ffmpeg_reader.py](https://localhost:8080/#) in VideoReader(filename, codec, pix_fmt, crop_xywh, resize, resize_keepratio, resize_keepratioalign)
125 resize_keepratioalign,
126 ):
--> 127 assert os.path.exists(filename) and os.path.isfile(
128 filename
129 ), f"{filename} not exists"
AssertionError: http://195.196.36.242/mjpg/video.mjpg not exists
best regards
Hey,
I started experimenting with ffmpegcv and have come across this issue where when I use a .ts (TransportStream) container, get_info crashes as the XML output from ffprobe does not contain the "nb_frames" field.
When using an mp4 container everything works as expected.
I should point out that I am using Ubuntu 18.04 with ffmpeg version 4.3.2 at the moment and have also tested with ffmpeg 6.0 running on Ubuntu 22.04.
Could you please add a check to only set the field in the video info if it exists in the xml output?
Thanks!
I want to write multiple videos at a same time using VideoWriterNV. My plan is to use multiprocessing.
My concern is that whether VideoWriterNV encoding works in multiple processes since there is only one NVENC per GPU.
If yes, will it achieve a proportional comsumption time? (i.e. 1+1+1≈3)
Plus, is there any built-in function in VideoWriterNV for parallel encoding that is more efficient that multiprocessing?
Thanks!
Hello, is there any way to jump directly to a know frame number? Something like cap.set(cv2.CAP_PROP_POS_FRAMES, frame)?
Thanks,
Gabor
Did a quick test
import ffmpegcv as fcv
cap = fcv.VideoCaptureStream("rtsp://admin:[email protected]:554/stream1", cv2.CAP_FFMPEG)
gives:
File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/__init__.py", line 398, in VideoCaptureStream
return FFmpegReaderStream.VideoReader(
File "/usr/local/bin/.virtualenvs/catdetector/lib/python3.8/site-packages/ffmpegcv/ffmpeg_reader_stream.py", line 20, in VideoReader
assert pix_fmt in ["rgb24", "bgr24", "yuv420p", "nv12", "gray"]
AssertionError
Stream is 720, x 1280 color h264. Works with cv2
cap = cv2.VideoCapture("rtsp://admin:[email protected]:554/stream1")
On OpenCV, one of the problems with VideoWriter
is that we lose audio as part of the process.
Wondering if ffmpegcv addresses this issue?
This is more of a TODO for someone, possibly myself .. looking to see the performance of reading a stream into a GPU surface vs the NVidia VPF option, which I think also uses FFMPEG pipe?
I run the following comparison code with the line profiler of python library. (Python version 3.8.5 / FFMPEG version 4.2.7-0ubuntu0.1)
In this scenario, we assume that all images (389 .png) are in the list with RGB order, each of which is a (high) resolution of 2160x3840. Therefore in make_video_ffmpegcv
, i assigned pix_fmt="rgb24"
. I also checked that the gpu memories are assigned during the running make_video_ffmpegcv
. Is there anything that i missed for the function make_video_ffmpegcv
?
Best,
ffmpeg,videowriter working fine 24/7 on 4 cameras.
Occasionally, once or twice a day, I get:
File ".../site-packages/ffmpegcv/ffmpeg_writer.py", line 70, in write
self.process.stdin.write(img)
BrokenPipeError: [Errno 32] Broken pipe
No pipes are involved in this operation, so it must be ffmpegcv's pipe
B
cap = ffmpegcv.VideoCaptureCAM("rtsp://", (1280, 720))
this line can not running.
In H264 encoded MTS files, ffmpegcv read twice frames than cv2.
Other programs (including Premere Pro and Windows) report same results as cv2, not ffmpegcv.
>>> import cv2
>>> import ffmpegcv
>>> capture_cv2 = cv2.VideoCapture("1-1.MTS")
>>> capture_cv2.get(cv2.CAP_PROP_FPS)
25.0
>>> capture_cv2.get(cv2.CAP_PROP_FRAME_COUNT)
15086.0
>>> capture_cv2.release()
>>> capture_ffmpegcv = ffmpegcv.VideoCapture("1-1.MTS")
>>> capture_ffmpegcv.fps
50.0
>>> capture_ffmpegcv.count
30134
>>> capture_ffmpegcv.release()
Hi, first of all thank you for this wonderful solution. I am trying to convert my exising cv2 writer to ffmpegcv but video is always 0 bytes no matter how i tried. Can you please point out what I did wrong? Thank you in advance. I understand the image width height can be automated but I just wanna stick to current setup for future changes. CV2 ran without problem.
frame = cv2.imread(os.path.join(image_folder, images[0]))
height, width, layers = frame.shape
#video = cv2.VideoWriter(video_name, fourcc, 15, (width, height))
video = ffmpegcv.VideoWriterNV(video_name, 'hevc_nvenc', 15, (width, height)) #40
for image in images:
img = cv2.imread(os.path.join(image_folder, image))
video.write(img)
video.release()
Do you plan to update the audio record feature also ?
Thansk
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.