On-device programming¶
While regular (firmware) on-device development is not possible due to closed nature of native tooling, we still expose a couple of alternative ways of running custom code:
Scripting - Using Python3.9 with Script node
Creating your own NN model to run more computationally heavy features
Creating custom OpenCL kernels
Note
With Series 3 OAK cameras, users will be able to run custom containirized apps on the OAK camera itself.
Using Script node¶
Using Script node allows you to run custom python scripts on the device itself, which allows users greater flexibility when constructing pipelines.
Script node is also very useful when using multiple neural networks in series and you need to process the output of the 1st one before feeding an image to the second one. Example here would be face age/gender recognition demo - first NN would detect faces, pass detections to the Script node which would create ImageManip configurations to crop the original frame and feed the face age/gender recognition NN only the cropped face frame.
For running computationally heavy features (eg. image filters), due to performance reasons you might want to avoid using Script node and rather go with one of the 2 options described below.
Creating custom NN models¶
You can create custom models with your favourite NN library, convert the model into OpenVINO and then compile it into the .blob.
More information on this topic can be found on Converting model to MyriadX blob documentation.
Refer to Run your own CV functions on-device page to find out more, or check On-device Pointcloud NN model tutorial.
Creating custom OpenCL kernel¶
Creating custom NN models has some limitations, for example unsupported layers by OpenVINO/VPU. To avoid these limitations, you could consider creating custom OpenCL kernel and compile it for the VPU. This kernel will run on SHAVE core(s) on the VPU. One should also take into the account that this option is not super user friendly. We plan on creating a tutorial on how to develop these and run them on OAK cameras.