need
Hybrid App development, native shell + webApp, calls the native camera function in the web part and displays the camera content in the designated area of the web page. At the same time, you can manually take pictures and perform face recognition, and display the recognition results on the web page.
technology stack
vue, Html5, video tag, Android, IOS, Baidu AI
analyze
1. Use navigator.mediaDevices.getUserMedia to call the system’s native camera function
2. The video tag displays the camera content
3. Obtain images using the canvas tag
4. Upload the image to the server and identify the image through Baidu AI
5. Web displays recognition results
core code
1. Call the system’s native camera function and use the video tag to display html:
<video id=webcam :style=videoStyle :width=videoWidth :height=videoHeight loop preload > </video>
JavaScript:
initVideo() { let that = this; this.video = document.getElementById(webcam); setTimeout(() => { if ( navigator.mediaDevices.getUserMedia || navigator.getUserMedia || navigator.webkitGetUserMedia || navigator.mozGetUserMedia ) { //Call the user media device and access the camera this.getUserMedia( { video: { width: { ideal: that.videoWidth, max: that.videoWidth }, height: { ideal: that.videoHeight, max: that.videoHeight }, facingMode: user, //Front camera frameRate: { ideal: 30, min: 10 } } }, this.videoSuccess, this.videoError ); } else { this.$toast.center(Camera failed to open, please check permission settings!); } }, 300); }, getUserMedia(constraints, success, error) { if (navigator.mediaDevices.getUserMedia) { //The latest standard API navigator.mediaDevices .getUserMedia(constraints) .then(success) .catch(error); } else if (navigator.webkitGetUserMedia) { //webkit core browser navigator.webkitGetUserMedia(constraints, success, error); } else if (navigator.mozGetUserMedia) { //firfox browser navigator.mozGetUserMedia(constraints, success, error); } else if (navigator.getUserMedia) { //Old API navigator .getUserMedia(constraints, success, error); } }, videoSuccess(stream) { this.mediaStreamTrack = stream; this.video.srcObject = stream; this.video.play(); }, videoError(error) { console.error(error); this.$toast.center(Camera is on Failed, please check permission settings!); },2. Canvas obtains camera pictures
JavaScript:
this.canvas = document.createElement(canvas); .... let context = this.canvas.getContext(2d); context.drawImage(this.video, 0, 0, this.videoWidth, this.videoHeight); this. imgSrc = this.canvas.toDataURL(image/png);
3. Call Baidu AI to identify pictures
JavaScript:
let that = this; let base64Data = this.canvas.toDataURL(); let blob = this.dataURItoBlob(base64Data); // var file = new FormData(); file.append(file, blob); file.append(key , that.uuid); util.ajax .post(XXXXXXXXXX, file, { headers: { Content-Type: multipart/form-data } }) .then(function(response) { if ((response.status = 200)) { .....recognition is successful, display the result} else { ....recognition failed} }) .catch(function(error) { console.error(error); }); //Convert base64 to Blob dataURItoBlob(base64Data) { var byteString; if (base64Data.split(,)[0].indexOf(base64) >= 0) byteString = atob(base64Data.split(,)[1]); else byteString = unescape(base64Data.split(,)[1]); var mimeString = base64Data .split(,)[0] .split(:)[1] .split(;)[0]; var ia = new Uint8Array(byteString.length); for (var i = 0; i < byteString.length; i++) { ia[i] = byteString.charCodeAt(i); } return new Blob([ia], { type : mimeString }); },Mobile phone adaptation
1. Since Android’s permission management has changed since Android 6, the Android native shell needs to be processed as follows:
myWebView.setWebChromeClient(new WebChromeClient() { @TargetApi(Build.VERSION_CODES.LOLLIPOP) @Override public void onPermissionRequest(final PermissionRequest request) { request.grant(request.getResources()); }});2. IOS system, available after Safari11
3. OverconstrainedError error. Some Android phones will report an OverconstrainedError error. The reason is that the camera parameter settings are unreasonable and the specified settings cannot be found.