[SURVEY RESULTS] The 2024 edition of State of Software Modernization market report is published!
GET IT here

How to Use React Native & OpenCV for Image Processing [2024]

readtime
Last updated on
July 3, 2024

A QUICK SUMMARY – FOR THE BUSY ONES

Image processing using OpenCV

What is OpenCV: Open Source Computer Vision Library (OpenCV) is an open-source software library for computer vision and machine learning. It offers over 2500 optimized algorithms for tasks like face detection, object identification, motion tracking, and more.

OpenCV and React Native: Though integrating OpenCV with React Native has some challenges, it's possible and useful for tasks that require image processing. Existing resources and libraries can help you achieve this integration.

Tutorial Overview: The tutorial guides you through building a React Native project that uses OpenCV to determine if a captured photo is clear or blurred. The steps are divided into Android and iOS setups, with code snippets provided for each.

TABLE OF CONTENTS

How to Use React Native & OpenCV for Image Processing [2024]

Introduction

OpenCV enables React Native developers to process images on mobile devices (most likely you’d like to process images taken by your device’s camera).

The top advantages are evident:

  • Easy to implement
  • Easy to use
  • Lots of tutorials all over the Internet and solid official documentation of OpenCV
  • The size of your mobile app will be only a dozen or so megabytes bigger

Let me show you how I used the OpenCV and React Native step by step to process my images, but first a few words of introduction.

What is OpenCV

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.

The library has more than 2500 optimized algorithms, including a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms.

These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, extract 3D models of objects, produce 3D point clouds from stereo cameras, stitch images together to produce a high resolution image of an entire scene.

It can also find similar images from an image database, remove red eyes from images taken using flash, follow eye movements, recognize scenery and establish markers to overlay it with augmented reality, etc.

OpenCV has more than 47k people community and an estimated number of downloads exceeding 14 million. The library is used extensively in companies, research groups and by governmental bodies. OpenCV is written natively in C++.

Since 2010 OpenCV was ported to the Android environment, it allows using the full power of the library in mobile applications development. In 2012 OpenCV development team actively worked on adding extended support for iOS. Full integration is available since version 2.4.2 (2012).

React Native was first released in 2015 by Facebook. React Native lets you build mobile apps using only JavaScript. It uses the same design as React, letting you compose a rich mobile UI from declarative components.

React Native and OpenCV are good friends

If we google for “react native opencv” we stumble upon the following links:

What is this tutorial about?

In this tutorial, we will be building an example project which uses the device’s camera to take a photo, processes it with native code and returns the information if the taken image is blurred or clear. Doing it in plain JavaScript would be highly ineffective. JavaScript is not sufficient for very heavy calculations.

Take a note that we are by no means Java or Objective-C developers hence our Java/Objective-C code may be far from good. If you want to contribute to this subject, please contact us.

React Native image processing using OpenCV - step by step

Here's what we're going to do:

Android Setup:

  • Import OpenCV to Android Studio
  • Update build.gradle files
  • Create required Java files and add permissions
  • Configure MainApplication.java and BaseLoaderCallback
  • Set up the OpenCV package

iOS Setup:

  • Add opencv2.framework to Xcode
  • Create and configure .pch file
  • Create RNOpenCvLibrary.h and RNOpenCvLibrary.mm files
  • Configure Info.plist

JavaScript Integration:

  • Set up a NativeModules folder and OpenCV.js file
  • Install third-party libraries
  • Use the camera and OpenCV for image processing
  • Check for blurry images using native functions

Now, let's go through these step in more detail.

OpenCV basic preparation

Step 1

react-native init reactNativeOpenCvTutorial

Step 2

Run downloadAndInsertOpenCV.sh script in your project’s catalog (can be found here) which downloads and inserts openCV files for both Android and iOS. The paths in the file might not match your preferences so you may need to change them.

Tutorial for Android

Step 1

Open your project in Android Studio.

Step 2

Follow the tips in Android Studio for synchronizing your project.

Step 3

Download the newest version of OpenCV for Android. In my case, it’s 3.4.1.

Step 4

Import OpenCV to Android Studio – from File -> New -> Import Module, choose sdk/java folder in the unzipped OpenCV archive.

Step 5

Update build.gradle under imported OpenCV module to update 4 fields to match your project’s build.gradle:

  • compileSdkVersion
  • buildToolsVersion
  • minSdkVersion
  • targetSdkVersion.

Step 6

Add module dependency:

Application -> Module Settings, and select the “Dependencies” tab. Click “+” icon at the bottom, choose “Module Dependency” and select the imported OpenCV module.

For Android Studio v1.2.2, to access Module Settings in the project view, right-click the dependent module -> Open Module Settings.

File manager screenshot number 1, setting up for OpenCV image processing.

Click on the “+”, select “Module dependency” and select the OpenCV library from the list.

File manager screenshot number 2, setting up for OpenCV image processing.

Step 7

Inside android/app/src/java create a package called i.e. “com.reactlibrary”.

Step 8

Update your Manifest with proper permissions:

Code block number 1, presents steps for OpenCV image processing.

See the whole file.

Step 9

Create a file RNOpenCvLibraryModule.java inside your newly created package and fill it as shown here.

Step 10

Create a file called RNOpenCvLibraryPackage.java inside your newly created package and fill it as shown here.

Step 11

Add proper imports in your MainApplication.java file, add your OpenCV package to the list and proper code to the MainApplication class like this:

Code block number 2, presents steps for OpenCV image processing.

Add new RNOpenCvLibraryPackage() to your list of packages.

Code block number 3, presents steps for OpenCV image processing.

Add BaseLoaderCallback in your MainApplication class:

Code block number 4, presents steps for OpenCV image processing.

Also, add following callbacks to your MainApplication class:

Code block number 5, presents steps for OpenCV image processing.

See the whole file here.

Tutorial for iOS

Step 1

Open the iOS project in XCode.

Step 2

Add opencv2.framework to your Linked Frameworks and Libraries.

File manager screenshot number 3, setting up for OpenCV image processing.

Step 3

Create a new group in the iOS catalog. I named it “OpenCV”.

Step 4

Add a .pch file and insert it into the OpenCV catalog.

Step 5

Add proper content to your .pch file – shown here.

Step 6

Create a file named RNOpenCvLibrary.h and fill it as shown here.

Step 7

Create a file named RNOpenCvLibrary.mm and fill it as shown here.

Step 8

Set “Precompile Prefix Header” to “Yes” and set “Prefix Header” path like this:

File manager screenshot number 4, setting up for OpenCV image processing.

Step 9

Add the following to your Info.plist file:

Code block number 6, presents steps for OpenCV image processing.

See the whole file.

Final part – JavaScript

Step 1

Inside your src folder create a folder named e.g. “NativeModules” and a file named OpenCV.js and fill it with:

Code block number 7, presents steps for OpenCV image processing.

See the whole file.

Step 2

We will be using some third-party libraries for quick setup. Open up the terminal and type:

Code block number 8, presents steps for OpenCV image processing.

Step 3

Don’t forget to link the libraries: react-native link.

We will be using the following file as a reference.

In line 126 we set up the camera and in line 135 we create a touchable element to handle taking photos. Taking photos is handled by the takePicture function. It takes a photo, saves the data in the local state and proceeds to check if the photo is blurred.

The function proceedWithCheckingBlurryImage uses native function checkForBlurryImage and returns a simple information if the photo is blurred or not.

Step 4

If you encounter build problems for Android connected to the camera, refer to my build.gradle file, especially maven { url “https://jitpack.io” } line and resync the project in Android Studio.

If you have errors related to Google Play services, check this issue on GitHub.

If you have errors related to react-native-camera, check the master branch by changing the version in package.json like this:

Code block number 9, presents steps for OpenCV image processing.

Example results

An example clear photo. We receive a preview of the photo and have the option to either use it or repeat it.

Sharp image as an effect of OpenCV image processing.

An example blurred photo. We receive a toast message informing us that the photo is blurred and have to repeat it.

Blurry image as an effect of OpenCV image processing.

Don’t forget to see the final product on our GitHub

References:

  1. Setting up OpenCV for Android in Android Studio (StackOverflow)
  2. Native modules for React Native (GitHub)
  3. Native Java code for checking if provided image is blurry (OpenCV official)
  4. Native Objective-C code for checking if provided image is blurry (StackOverflow)
  5. OpenCV’s official website (OpenCV official)

Frequently Asked Questions

No items found.

Our promise

Every year, Brainhub helps 750,000+ founders, leaders and software engineers make smart tech decisions. We earn that trust by openly sharing our insights based on practical software engineering experience.

Authors

Piotr Suwała
github
JavaScript Software Engineer

Full-stack software engineer with 8 years of professional experience. Passionate of React.js and React Native. Graduate of The Silesian University of Technology.

Bianka Pluszczewska
github
Tech Editor

Software development enthusiast with 8 years of professional experience in this industry.

Piotr Suwała
github
JavaScript Software Engineer

Full-stack software engineer with 8 years of professional experience. Passionate of React.js and React Native. Graduate of The Silesian University of Technology.

Bianka Pluszczewska
github
Tech Editor

Software development enthusiast with 8 years of professional experience in this industry.

Read next

No items found...