Well, at end of Part IV. I was planning to deal with the different type of Coral boards, but something more interesting popped up. During my last 'outdoor' experiment, I realized that it would be nice to have an ok camera attached to the Coral board, and as lately I was reading a lot about the Raspberry Pi High Quality Camera, I purchased one.
The camera has a 12.3 megapixel Sony IMX477 sensor, and 2 lenses:
... my original plan was to connect this camera directly to the Coral Board, but it quickly turned out, I was not doing my homework, as I haven't read the documentations carefully in advance. Amongst the Coral board Features there is 24-pin FFC connector for MIPI CSI-2 camera (4-lane) listed, this is where I connected the Camera before. My idea was that the Pi camera just should work fine, but the Pi Camera has a 2-lane MIPI CSI camera port ... and that was before I realized that there are specific apps like raspistill and raspicam to handle the cam! So there was no other way, but to get a Raspberry Pi 4.
With the Raspberry Pi 4 on my desk, I got the cam working in no time. Next task was to get a stream up from the Pi to my machine, where I could do some test with the Coral USB accelerator. To implement this raspivid is piped into cvlc, which will initiate an RTSP stream on port 8554:
# raspivid -o - -t 0 -vf -w 640 -h 480 -fps 30| cvlc -vvv stream:///dev/stdin --sout '#rtp{sdp=rtsp://:8554/}' :demux=h264
On the receiving end (my machine), I modified my python server, so it can receive RTSP:
if(rtsp):
os.environ["OPENCV_FFMPEG_CAPTURE_OPTIONS"] = "rtsp_transport;udp"
self.cap = cv2.VideoCapture(rtsp)
OpenCV is using FFMpeg under the hood, and unless I set the CAPTURE_OPTION = rtsp_transport;udp where udp is the important thing, I was not getting anything. This way the streaming works, but it has a pretty bad delay! I started to play around with different streaming settings but haven't found a decent one yet. Best speed so far is given, when I stream using netcat from the Pi toward my laptop (thor) on port 65000
# raspivid -o - -t 0 -vf -w 640 -h 480 -fps 30 | netcat thor.local 65000and on my laptop I listen to port 65000 and pipe it to mpv (mplayer)
# nc -l 65000 | mpv --no-correct-pts --fps=300 --demuxer=lavf -In this case there is no delay, and the feed is near real time ... so it must be possible to get the same speed with OpenCV, I just have no more free time left for the weekend to figure it out how ...
As you can see on the pictures below, the cam would require some case and a tripod, but instead I just 3d printed an "L" shaped holder. It's stable enough so even with the heavy 16 mm lens, it won't tilt. Last picture demonstrates the "evolution" of the holder, to the final shape, which provides enough stability and space to adjust the camera.
I'm pretty happy with my new setup, it opens up a lot of new possibilities. Guess I have to fix the delay with OpenCV, but hopefully that should be 'easy'.
