Our team is working on an Android application where we need to match multiple objects from a camera feed with a custom backend. The backend has a database of different images and their respective AR effects, such as videos or 3D models. When an object/image is detected in the camera feed, we want to display the corresponding AR effect on that object/image in the camera feed.
Android App Side:
- How can we achieve this on the Android app side?
- What kind of frameworks or libraries should we use to:
- Capture the camera feed?
- Send the camera feed to the backend for processing.
- Receive the results from the backend?
- Display AR effects (videos, 3D models) on the detected objects/images.
Custom Backend Side:
- How can we efficiently match an object coming in the camera frame with images stored in the database?
- How do we associate each matched object with the respective AR effect (video or 3D model)?
- What API protocols should we expose to the Android app to send the camera feed, receive results, and get the AR effect with its position relative to the marker/respective image?
We tried ARCore, and some REST APIS, etc, but the results have not met yet.
New contributor
Zaheer UD Din Baber is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.