esp32-cam
esp32-cam is a arduino based board with a builtin camera along with esp32 chipset.
PIN diagram of esp32-CAM
?
Configure Board Manager
Edit the configuration file, to add an entry in the additional_urls list under board_manager.
prabu@homepc2 ~> cat ~/.arduino15/arduino-cli.yaml
board_manager:
additional_urls:
# []
- http://arduino.esp8266.com/stable/package_esp8266com_index.json
- https://espressif.github.io/arduino-esp32/package_esp32_index.json
Download and Install cores
Update the index of cores. You should see that the ESP32 index has been downloaded.
prabu@homepc2 ~> arduino-cli core update-index
Downloading index: package_index.tar.bz2 downloaded
Downloading index: package_esp8266com_index.json downloaded
Downloading index: package_esp32_index.json downloaded
Install the core. The command automatically downloads the packages
prabu@homepc2 ~> arduino-cli core install esp32:esp32
Downloading packages...
esp32:esp32-arduino-libs@idf-release_v5.1-442a798083 42.41 MiB / 356.11 MiB [===>--------------------------] 11.91% 04m52s
Around 1GB of new packages got installed in home/prabu.arduino15/staging/packages/
Identify FQBN and port
As per the website cytron given below, the board FQBN should be esp32:esp32:esp32wrover for the board model number HW-818. However the pin configuration is not matching with the board and the chip is only ESP32-S. So decided to use esp32cam as the pin and chip matches with AI Thinker board but has an extra usb.
prabu@homepc2 ~> arduino-cli board list
Port Protocol Type Board Name FQBN Core
192.168.1.231 network Network Port NodeMCU 1.0 (ESP-12E Module) esp8266:esp8266:nodemcuv2 esp8266:esp8266
/dev/ttyS0 serial Serial Port Unknown
/dev/ttyUSB0 serial Serial Port (USB) Unknown
prabu@homepc2 ~> arduino-cli board listall |grep 'AI Thinker ESP32-CAM'
AI Thinker ESP32-CAM esp32:esp32:esp32cam
https://www.cytron.io/tutorial/getting-started-with-esp32-cam↗
Flashing mode in ESP32-CAM
Important: GPIO 0 needs to be connected to GND so that you’re able to upload code.
GPIO 0 determines whether the ESP32 is in flashing mode or not. This GPIO is internally connected to a pull-up 10k Ohm resistor.
When GPIO 0 is connected to GND, the ESP32 goes into flashing mode and you can upload code to the board.
To make the ESP32 run “normally”, you just need to disconnect GPIO 0 from GND.
Compile and Upload code
prabu@homepc2 ~/sketches (master)> arduino-cli compile --fqbn esp32:esp32:esp32cam blink
prabu@homepc2 ~/sketches (master)> arduino-cli compile --upload --fqbn esp32:esp32:esp32cam --port /dev/ttyUSB0 blink
Serial Communication
Serial monitoring can be done using arduino-cli or special tool like picocom.
prabu@homepc2 ~> arduino-cli monitor -p /dev/ttyUSB0 --config baudrate=115200
Monitor port settings:
baudrate=115200
Connected to /dev/ttyUSB0! Press CTRL-C to exit.
prabu@homepc2 ~> picocom -b 115200 /dev/ttyUSB0
Factory Reset:
To perform a factory reset on the ESP32-CAM, you generally need to erase its flash memory. This process will remove all the data and settings stored on the ESP32, including any sketches you have uploaded. Here are the steps you can follow:
Erase Flash Memory: Use the following command to erase the flash memory on your ESP32-CAM:
esptool.py --chip esp32 erase_flash
prabu@homepc2 ~/s/CameraSend (master)> esptool.py --chip esp3
2 erase_flash
esptool.py v4.7.0
Found 2 serial ports
Serial port /dev/ttyUSB0
Connecting...........
Chip is ESP32-D0WDQ6-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
WARNING: Detected crystal freq 41.01MHz is quite different to normalized freq 40MHz. Unsupported crystal in use?
Crystal is 40MHz
MAC: 34:98:7a:b6:28:cc
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 9.9s
Hard resetting via RTS pin...
Additional Notes
Boot Mode: Ensure that your ESP32-CAM is in boot mode when you are performing these steps. This usually involves holding the BOOT button (if available) or connecting GPIO 0 to GND while pressing the RESET button.
Power Cycle: After erasing the flash, you may need to power cycle the ESP32-CAM by disconnecting and reconnecting the power.
Reset Camera sensor settings
// Reset sensor settings
sensor_t *s = esp_camera_sensor_get();
if (s) {
s->reset(s);
Serial.println("Camera sensor settings reset");
} else {
Serial.println("Failed to get camera sensor settings");
}
mpv
The following command gives a stutter
$mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample --fullscreen --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1
When using mpv to directly watch a stream and you want to maximize playback speed while achieving high quality, there are several options and configurations you can adjust. Here’s how you can optimize mpv for both speed and quality: Command Line Options
Hardware Decoding: Enable hardware decoding to offload decoding tasks to the GPU, which can significantly improve playback performance.
mpv --hwdec=auto rtsp://192.168.1.105:554/mjpeg/1
High Quality Video Output: Use the gpu video output driver with high quality settings.
mpv --vo=gpu --profile=gpu-hq rtsp://192.168.1.105:554/mjpeg/1
Cache Settings: Increase the cache size to ensure smooth playback, especially for network streams.
mpv --cache=yes --cache-secs=60 rtsp://192.168.1.105:554/mjpeg/1
Video Sync: Adjust video sync method for smoother playback.
mpv --video-sync=display-resample rtsp://192.168.1.105:554/mjpeg/1
Example Command
Combining the above options into a single command:
mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample rtsp://192.168.1.105:554/mjpeg/1
Additional Tips
Fullscreen Mode: Start in fullscreen mode for better performance on some systems.
mpv --fullscreen rtsp://192.168.1.105:554/mjpeg/1
Frame Dropping: If you experience stuttering, you can allow frame dropping to ensure smoother playback.
mpv --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1
Verbose Output: For troubleshooting performance issues, use verbose output to monitor what’s happening.
mpv --msg-level=all=v rtsp://192.168.1.105:554/mjpeg/1
Summary
Here’s a consolidated command that includes all the optimizations mentioned:
mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample --fullscreen --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1
These settings should help you maximize playback speed and achieve high quality when watching an RTSP stream using mpv. Adjust the settings as needed based on your system’s performance and the quality of the stream.
ffmpeg
$ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output.mp4
recording data:
frame= 2248 fps=6.3 q=24.0 size= 135168KiB time=00:05:59.36 bitrate=3081.3kbits/s dup=1271 drop=0 speed= 1x
$ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output_optimized.mp4
recording data:
frame= 1889 fps=6.3 q=24.0 size= 115200KiB time=00:05:01.92 bitrate=3125.7kbits/s dup=1047 drop=0 speed=1.01x
ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 28 -an output_low_size.mp4
recording data:
frame= 1448 fps=6.5 q=-1.0 Lsize= 50742KiB time=00:03:51.36 bitrate=1796.7kbits/s dup=780 drop=0 speed=1.03x
ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx265 -preset fast -crf 28 -c:a copy output_HEVC.mp4
frame= 2459 fps=6.4 q=29.9 Lsize= 44751KiB time=00:06:33.12 bitrate= 932.5kbits/s dup=1311 drop=0 speed=1.02x
prabu@homepc2 ~ [255]> ls -lh output.mp4
-rw-r--r-- 1 prabu prabu 174M Jun 29 11:15 output.mp4
prabu@homepc2 ~> ls -lh *.mp4
-rw-r--r-- 1 prabu prabu 50M Jun 29 11:32 output_low_size.mp4
-rw-r--r-- 1 prabu prabu 44M Jun 29 11:43 output_HEVC.mp4
-rw-r--r-- 1 prabu prabu 133M Jun 29 11:05 default_output.mp4
-rw-r--r-- 1 prabu prabu 174M Jun 29 11:15 output.mp4
-rw-r--r-- 1 prabu prabu 130M Jun 29 11:26 output_optimized.mp4
prabu@homepc2 ~> mpv output.mp4
(+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:02:26 / 00:07:50 (31%)
Exiting... (Quit)
prabu@homepc2 ~ [4]> mpv default_output.mp4
(+) Video --vid=1 (*) (mjpeg 1600x1200 2.876fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:02 / 00:04:50 (1%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_optimized.mp4
(+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:20 / 00:05:44 (6%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_low_size.mp4
(+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:25 / 00:03:51 (11%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_HEVC.mp4
(+) Video --vid=1 (*) (hevc 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuv422p
V: 00:00:08 / 00:06:33 (2%)
The output you provided from ffmpeg gives us some key metrics about the transcoded video:
Frame: Indicates the number of frames processed. FPS: Frames per second processed. Q: Quantizer scale (quality measure). Size: Size of the output video file (16384KiB or approximately 16 GB). Time: Duration of the video (00:00:42.08 or 42 seconds). Bitrate: Average bitrate of the video (3189.6kbits/s).
Comparing with Raspberry Pi Zero 2W
The Raspberry Pi Zero 2W is less powerful compared to other models like the Raspberry Pi 3 or 4. It uses a single-core ARM Cortex-A53 processor running at 1 GHz and has 512 MB of RAM. Transcoding Performance Considerations
When using ffmpeg for video transcoding on a Raspberry Pi Zero 2W, several factors will affect performance and achievable video quality:
CPU Performance: The single-core ARM processor on the Pi Zero 2W is less powerful compared to multi-core processors on other models. This limitation impacts the encoding speed (speed=1.04x in your case) and the ability to handle real-time transcoding of high-resolution video streams.
Video Resolution: Higher resolutions require more processing power. If your RTSP stream is at a higher resolution (e.g., 1080p), the Pi Zero 2W may struggle to transcode in real-time at acceptable quality and frame rates.
Codec Efficiency: H.264 encoding (used in your command) is less demanding than H.265 (HEVC). Choosing a more efficient codec like H.264 helps in reducing the processing load on the Pi Zero 2W.
Estimated Performance on Raspberry Pi Zero 2W
Given the specifications and assuming similar video input (RTSP stream), the transcoding performance on a Raspberry Pi Zero 2W might yield:
Lower FPS: Expect lower frame rates compared to your current setup (fps=6.5). Increased Encoding Time: Longer time to process each frame (speed=1.04x suggests it’s close to real-time). Lower Bitrate: Depending on the encoding settings and resolution, the bitrate might decrease to accommodate the processing capability of the Pi Zero 2W.
Recommendations
To optimize performance on the Raspberry Pi Zero 2W:
Lower Resolution: Reduce the resolution of the transcoded video (-vf scale=W:H) to match the Pi Zero 2W’s processing capabilities. Codec Choice: Stick with H.264 or test H.265 with conservative settings (-c:v libx265 -preset fast -crf 28) to balance quality and performance. Monitoring Performance: Use htop or top commands on the Pi Zero 2W to monitor CPU usage during transcoding to ensure it stays within acceptable limits.
Adjusting these parameters will help achieve a balance between video quality and transcoding performance on the Raspberry Pi Zero 2W.
To optimize the file size while maintaining acceptable video quality:
Adjusting -crf and -preset: Fine-tune the -crf (Constant Rate Factor) and -preset options in ffmpeg to find a balance between video quality and file size. For example:
bash
ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output_optimized.mp4
Decrease the -crf value for higher quality (larger file size) or increase it for smaller file size (lower quality). Experiment with different -preset options (fast, medium, slow) to optimize encoding speed versus compression efficiency.
Resolution and Frame Rate: Consider reducing the resolution (-vf scale=W:H) or frame rate (-r) if maintaining high resolution and frame rate is not critical for your application.
Monitoring: Use mpv or other media players to visually inspect the quality of the output and adjust encoding parameters accordingly.
By iterating through these optimizations, you can achieve a balance between video file size and quality that meets your specific requirements for playback and storage.
© Prabu Anand K 2020-2026