esp32-cam

esp32-cam is a arduino based board with a builtin camera along with esp32 chipset.

PIN diagram of esp32-CAM

?

Configure Board Manager

Edit the configuration file, to add an entry in the additional_urls list under board_manager.

prabu@homepc2 ~> cat ~/.arduino15/arduino-cli.yaml
board_manager:
   additional_urls:
      # []
      - http://arduino.esp8266.com/stable/package_esp8266com_index.json
      - https://espressif.github.io/arduino-esp32/package_esp32_index.json

Download and Install cores

Update the index of cores. You should see that the ESP32 index has been downloaded.

prabu@homepc2 ~> arduino-cli core update-index
Downloading index: package_index.tar.bz2 downloaded
Downloading index: package_esp8266com_index.json downloaded
Downloading index: package_esp32_index.json downloaded

Install the core. The command automatically downloads the packages

prabu@homepc2 ~> arduino-cli core install esp32:esp32
Downloading packages...
esp32:esp32-arduino-libs@idf-release_v5.1-442a798083 42.41 MiB / 356.11 MiB [===>--------------------------]  11.91% 04m52s

Around 1GB of new packages got installed in home/prabu.arduino15/staging/packages/

Identify FQBN and port

As per the website cytron given below, the board FQBN should be esp32:esp32:esp32wrover for the board model number HW-818. However the pin configuration is not matching with the board and the chip is only ESP32-S. So decided to use esp32cam as the pin and chip matches with AI Thinker board but has an extra usb.

prabu@homepc2 ~> arduino-cli board list
Port          Protocol Type              Board Name                   FQBN                      Core
192.168.1.231 network  Network Port      NodeMCU 1.0 (ESP-12E Module) esp8266:esp8266:nodemcuv2 esp8266:esp8266
/dev/ttyS0    serial   Serial Port       Unknown
/dev/ttyUSB0  serial   Serial Port (USB) Unknown
prabu@homepc2 ~> arduino-cli board listall |grep 'AI Thinker ESP32-CAM'
AI Thinker ESP32-CAM                               esp32:esp32:esp32cam

https://www.cytron.io/tutorial/getting-started-with-esp32-cam ↗

Flashing mode in ESP32-CAM

Important: GPIO 0 needs to be connected to GND so that you’re able to upload code.

GPIO 0 determines whether the ESP32 is in flashing mode or not. This GPIO is internally connected to a pull-up 10k Ohm resistor.

When GPIO 0 is connected to GND, the ESP32 goes into flashing mode and you can upload code to the board.

To make the ESP32 run “normally”, you just need to disconnect GPIO 0 from GND.

Compile and Upload code

prabu@homepc2 ~/sketches (master)> arduino-cli compile --fqbn esp32:esp32:esp32cam blink
prabu@homepc2 ~/sketches (master)> arduino-cli compile --upload --fqbn esp32:esp32:esp32cam --port /dev/ttyUSB0 blink

Serial Communication

Serial monitoring can be done using arduino-cli or special tool like picocom.

prabu@homepc2 ~> arduino-cli monitor -p /dev/ttyUSB0 --config baudrate=115200
Monitor port settings:
baudrate=115200
Connected to /dev/ttyUSB0! Press CTRL-C to exit.
prabu@homepc2 ~> picocom -b 115200 /dev/ttyUSB0

Factory Reset:

To perform a factory reset on the ESP32-CAM, you generally need to erase its flash memory. This process will remove all the data and settings stored on the ESP32, including any sketches you have uploaded. Here are the steps you can follow:

Erase Flash Memory: Use the following command to erase the flash memory on your ESP32-CAM:

esptool.py --chip esp32 erase_flash
prabu@homepc2 ~/s/CameraSend (master)> esptool.py --chip esp3
2 erase_flash
esptool.py v4.7.0
Found 2 serial ports
Serial port /dev/ttyUSB0
Connecting...........
Chip is ESP32-D0WDQ6-V3 (revision v3.1)
Features: WiFi, BT, Dual Core, 240MHz, VRef calibration in efuse, Coding Scheme None
WARNING: Detected crystal freq 41.01MHz is quite different to normalized freq 40MHz. Unsupported crystal in use?
Crystal is 40MHz
MAC: 34:98:7a:b6:28:cc
Uploading stub...
Running stub...
Stub running...
Erasing flash (this may take a while)...
Chip erase completed successfully in 9.9s
Hard resetting via RTS pin...

Additional Notes

Boot Mode: Ensure that your ESP32-CAM is in boot mode when you are performing these steps. This usually involves holding the BOOT button (if available) or connecting GPIO 0 to GND while pressing the RESET button.

Power Cycle: After erasing the flash, you may need to power cycle the ESP32-CAM by disconnecting and reconnecting the power.

Reset Camera sensor settings

// Reset sensor settings
  sensor_t *s = esp_camera_sensor_get();
  if (s) {
    s->reset(s);
    Serial.println("Camera sensor settings reset");
  } else {
    Serial.println("Failed to get camera sensor settings");
  }

mpv

The following command gives a stutter

$mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample --fullscreen --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1

When using mpv to directly watch a stream and you want to maximize playback speed while achieving high quality, there are several options and configurations you can adjust. Here’s how you can optimize mpv for both speed and quality: Command Line Options

Hardware Decoding: Enable hardware decoding to offload decoding tasks to the GPU, which can significantly improve playback performance.

mpv --hwdec=auto rtsp://192.168.1.105:554/mjpeg/1

High Quality Video Output: Use the gpu video output driver with high quality settings.

mpv --vo=gpu --profile=gpu-hq rtsp://192.168.1.105:554/mjpeg/1

Cache Settings: Increase the cache size to ensure smooth playback, especially for network streams.

mpv --cache=yes --cache-secs=60 rtsp://192.168.1.105:554/mjpeg/1

Video Sync: Adjust video sync method for smoother playback.

mpv --video-sync=display-resample rtsp://192.168.1.105:554/mjpeg/1

Example Command

Combining the above options into a single command:

mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample rtsp://192.168.1.105:554/mjpeg/1

Additional Tips

Fullscreen Mode: Start in fullscreen mode for better performance on some systems.

mpv --fullscreen rtsp://192.168.1.105:554/mjpeg/1

Frame Dropping: If you experience stuttering, you can allow frame dropping to ensure smoother playback.

mpv --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1

Verbose Output: For troubleshooting performance issues, use verbose output to monitor what’s happening.

mpv --msg-level=all=v rtsp://192.168.1.105:554/mjpeg/1

Summary

Here’s a consolidated command that includes all the optimizations mentioned:

mpv --hwdec=auto --vo=gpu --profile=gpu-hq --cache=yes --cache-secs=60 --video-sync=display-resample --fullscreen --framedrop=vo rtsp://192.168.1.105:554/mjpeg/1

These settings should help you maximize playback speed and achieve high quality when watching an RTSP stream using mpv. Adjust the settings as needed based on your system’s performance and the quality of the stream.

ffmpeg

$ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output.mp4
recording data:
frame= 2248 fps=6.3 q=24.0 size=  135168KiB time=00:05:59.36 bitrate=3081.3kbits/s dup=1271 drop=0 speed=   1x

$ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output_optimized.mp4
recording data:
frame= 1889 fps=6.3 q=24.0 size=  115200KiB time=00:05:01.92 bitrate=3125.7kbits/s dup=1047 drop=0 speed=1.01x

ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 28 -an output_low_size.mp4
recording data:
frame= 1448 fps=6.5 q=-1.0 Lsize=   50742KiB time=00:03:51.36 bitrate=1796.7kbits/s dup=780 drop=0 speed=1.03x

ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx265 -preset fast -crf 28 -c:a copy output_HEVC.mp4
frame= 2459 fps=6.4 q=29.9 Lsize=   44751KiB time=00:06:33.12 bitrate= 932.5kbits/s dup=1311 drop=0 speed=1.02x

prabu@homepc2 ~ [255]> ls -lh output.mp4
-rw-r--r-- 1 prabu prabu 174M Jun 29 11:15 output.mp4
prabu@homepc2 ~> ls -lh *.mp4
-rw-r--r-- 1 prabu prabu  50M Jun 29 11:32 output_low_size.mp4
-rw-r--r-- 1 prabu prabu  44M Jun 29 11:43 output_HEVC.mp4
-rw-r--r-- 1 prabu prabu 133M Jun 29 11:05 default_output.mp4
-rw-r--r-- 1 prabu prabu 174M Jun 29 11:15 output.mp4
-rw-r--r-- 1 prabu prabu 130M Jun 29 11:26 output_optimized.mp4

prabu@homepc2 ~> mpv output.mp4
 (+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:02:26 / 00:07:50 (31%)
Exiting... (Quit)
prabu@homepc2 ~ [4]> mpv default_output.mp4
 (+) Video --vid=1 (*) (mjpeg 1600x1200 2.876fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:02 / 00:04:50 (1%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_optimized.mp4
 (+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:20 / 00:05:44 (6%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_low_size.mp4
 (+) Video --vid=1 (*) (h264 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuvj422p
V: 00:00:25 / 00:03:51 (11%)
Exiting... (Quit)
prabu@homepc2 ~> mpv output_HEVC.mp4
 (+) Video --vid=1 (*) (hevc 1600x1200 6.250fps)
VO: [gpu] 1600x1200 yuv422p
V: 00:00:08 / 00:06:33 (2%)

The output you provided from ffmpeg gives us some key metrics about the transcoded video:

Frame: Indicates the number of frames processed. FPS: Frames per second processed. Q: Quantizer scale (quality measure). Size: Size of the output video file (16384KiB or approximately 16 GB). Time: Duration of the video (00:00:42.08 or 42 seconds). Bitrate: Average bitrate of the video (3189.6kbits/s).

Comparing with Raspberry Pi Zero 2W

The Raspberry Pi Zero 2W is less powerful compared to other models like the Raspberry Pi 3 or 4. It uses a single-core ARM Cortex-A53 processor running at 1 GHz and has 512 MB of RAM. Transcoding Performance Considerations

When using ffmpeg for video transcoding on a Raspberry Pi Zero 2W, several factors will affect performance and achievable video quality:

CPU Performance: The single-core ARM processor on the Pi Zero 2W is less powerful compared to multi-core processors on other models. This limitation impacts the encoding speed (speed=1.04x in your case) and the ability to handle real-time transcoding of high-resolution video streams.

Video Resolution: Higher resolutions require more processing power. If your RTSP stream is at a higher resolution (e.g., 1080p), the Pi Zero 2W may struggle to transcode in real-time at acceptable quality and frame rates.

Codec Efficiency: H.264 encoding (used in your command) is less demanding than H.265 (HEVC). Choosing a more efficient codec like H.264 helps in reducing the processing load on the Pi Zero 2W.

Estimated Performance on Raspberry Pi Zero 2W

Given the specifications and assuming similar video input (RTSP stream), the transcoding performance on a Raspberry Pi Zero 2W might yield:

Lower FPS: Expect lower frame rates compared to your current setup (fps=6.5). Increased Encoding Time: Longer time to process each frame (speed=1.04x suggests it’s close to real-time). Lower Bitrate: Depending on the encoding settings and resolution, the bitrate might decrease to accommodate the processing capability of the Pi Zero 2W.

Recommendations

To optimize performance on the Raspberry Pi Zero 2W:

Lower Resolution: Reduce the resolution of the transcoded video (-vf scale=W:H) to match the Pi Zero 2W’s processing capabilities. Codec Choice: Stick with H.264 or test H.265 with conservative settings (-c:v libx265 -preset fast -crf 28) to balance quality and performance. Monitoring Performance: Use htop or top commands on the Pi Zero 2W to monitor CPU usage during transcoding to ensure it stays within acceptable limits.

Adjusting these parameters will help achieve a balance between video quality and transcoding performance on the Raspberry Pi Zero 2W.

To optimize the file size while maintaining acceptable video quality:

Adjusting -crf and -preset: Fine-tune the -crf (Constant Rate Factor) and -preset options in ffmpeg to find a balance between video quality and file size. For example:

bash

ffmpeg -i rtsp://192.168.1.105:554/mjpeg/1 -c:v libx264 -preset fast -crf 23 -an output_optimized.mp4

Decrease the -crf value for higher quality (larger file size) or increase it for smaller file size (lower quality). Experiment with different -preset options (fast, medium, slow) to optimize encoding speed versus compression efficiency.

Resolution and Frame Rate: Consider reducing the resolution (-vf scale=W:H) or frame rate (-r) if maintaining high resolution and frame rate is not critical for your application.

Monitoring: Use mpv or other media players to visually inspect the quality of the output and adjust encoding parameters accordingly.

By iterating through these optimizations, you can achieve a balance between video file size and quality that meets your specific requirements for playback and storage.