Down the rabbit hole, into the world of OpenCV..

Sprint 13

Up till now, my main points of focus in the FYP have shifted from one to the other in the following progression – design and development of Scope’s User Experience, development on a Visual Studio version of the app that runs a specific task, and testing of ScopeMate (a simplified version of Scope that ran Tesseract with basic pre-processing on-board).

A discussion among the team made us realise that the app requires a further restructuring of its workflow.

Scope Workflow v2.0

With this new workflow in mind, I decided to focus my energy more towards developing the Template Matching algorithm. To do this, I had to get familiar with OpenCV..

OpenCV

OpenCV is a cross-platform library of programming functions mainly aimed at real-time computer vision, developed byIntel, and now supported by Willow Garage and Itseez. It is free for use under the open source BSD license. It focuses mainly on real-time image processing. We are using some of the  OpenCV functions in the pre-processing stage of Scope.

Some examples of the functions of the OpenCV libraries we’re using include – Brightness, Contrast, Gray-scale, Template Matching, and Thresholding.

Template Matching

Template Matching is a technique in digital image processing for finding small parts of an image which match a template image. In Scope, template matching serves the purpose of identifying whether the given card is actually an NUS card by verifying the picture with a template of the NUS logo and comparing it to the picture. As such, this function serves as a check for the app to find out whether the card is an NUS business card or not.

This information will help accelerate the rate of processing and improve the accuracy of the results obtained from the OCR engine.

Hurdles

Being entirely new to OpenCV, it is taking me time to understand the purpose and  working of the various functions in the libraries. Furthermore, I had to apply these functions in the development of an Android application. This has proved to be a challenging yet rewarding learning process at the same time. While there are compilation errors that show up every now and then, working on this part has been fun because not only is it building my understanding of OpenCV, it’s also reinforcing my knowledge of Android development.

Looking forward to developing the template matching function to its entirety by the next week!

Getting hands dirty with code blocks and wireframes..

Sprint 2

In this Sprint, I dealt with Visual Studio coding environment in efforts of developing an eventual WPF application that would become the base for the desktop version of the app. The complete ecosystem of our app consists of the mobile app, a cloud interface to sync all the info to, and a desktop app with enhanced functions. I took up the task of developing the desktop app, and in this sprint I experimented with the Tesseract engine and its application in a WPF app.

Many tests and iterations of the code have yielded some promise, but the main issue I’m still facing is the error being thrown by the system about the Tesseract’s illicit access of some restricted memory. After breaking my head over this problem and trawling all over the internet about this issue, the problem turned out to be a persistent one. The most immediate solution is to repack the entire Tesseract engine library in a .NET wrapper, which acts as a bridge between the native code of the engine and enables the WPF app to call certain APIs.

So far, I’ve researched on how to carry this out and will start the process of packaging in the coming days.

Sprint 1

Sprint 1 saw me brainstorming, conceptualising and wireframing different UI designs for the app. This process required a  brainstorm and denotation of features and functionality that were available to the user. Eventually, it boiled down to 3 different designs. These were then examined and discussed upon by the team and we decided on one of them.

I had to revisit the wireframes after the discussion so as to comply with the Android User Experience Guidelines. The folks at Android have released some design principles and guidelines on designing the user interface and experience of a mobile app, and it is our imperative to conceive and materialise our app as per the guidelines given.

First steps into the waters..

So I’ve been given the opportunity to work with an amazing team of guys – Aravindh, Salman and Varun – on a pretty interesting project.

The task – Develop an Optical Character Recognition engine and make a useful application with it.

The tools given – Tesseract, an open-sourced OCR engine invented by HP and developed by Google.

Our aim – A mobile app that incorporates this engine to read contact cards and save the information on the device, and in the cloud.

A tough task, a high target, an incredible team – all the right ingredients are in place.

Time to make the magic happen.

 

 

We’ve decided to break down our tasks and achievements for every week into Sprints. Each Sprint covers a the coming week’s objectives  to be covered.

Sprint O

In this Sprint, I was tasked with learning the C# and ASP.NET programming languages from scratch. Although I have prior experience working with C and C++, C# and ASP.NET are whole new languages with different type of functionality and I had to conquer this in order to be able to work towards tinkering with the Tesseract engine and designing the Cloud web interface. We had decided to work towards developing an app for the Windows Phone OS and my job as desginer of the app required that I brush up on using Expression Blend too. Also, I was tasked with composing and compiling the Literature Review about existing OCR technologies and Tesseract in itself.

In short, the first week was mainly a week of research and learning. Looking forwarrd to the next Sprint.

Final Year Pensiveness

I’ve arrived at the last leg in my engineering degree. A point where all the prior years of teachings suddenly seem to make some sense in the bigger picture. A point in time where I’m looking towards things like getting a job, leaving my mark at the university (figuratively, that is) etc..

But first,  the Final Year Project. Detailed below is my progress in working with a great team in what seems to be a exciting project. Let’s hope the outcome turns out to be as awesome as I hope it is..