Home‎ > ‎

A Multi View Imaging (MVI) Processing Platform: Real Time Panoramic Mosaic Generation

"A Multi View Imaging (MVI) Processing Platform: Real Time Panoramic Mosaic Generation"  is an ICT R&D funded project. The project shall be carried out at VISpro Lab SEECS-NUST in collaboration with Epic Technologies Islamabad.

Project Abstract: Multi View Imaging (MVI) techniques are finding extensive usage in future multimedia, surveillance, inspection and video conferencing applications. Particularly, the multi-billion entertainment/film industry is spending huge amounts in R & D related to immersive multimedia (e.g. Immersive TV). However, the biggest hurdle to-date has been the parallel processing of multiple video streams acquired in real time. This limits the benefits of MVI based systems for most applications. The project aims to design and develop a prototype system aimed as a platform for Multi View Imaging (MVI) applications such as Immersive Television and Video Conferencing. The project will result in a complete embedded hardware system, capable of generating real time panoramic mosaics, using video input from multiple cameras.

Project Highlights ||  Project Start Date : Feb 2012 | Project Funding : PKR 14.89 Million  | Duration 20 Months | Extended till May 2014, Project closure  under process

Executive summary

Multi View Imaging (MVI) techniques are finding extensive usage in future multimedia, surveillance, inspection and video conferencing applications. Particularly, the multibillion entertainment/film industry is spending huge amounts in R & D related to immersive multimedia (e.g. Immersive TV). However, the biggest hurdle to-date has been the parallel processing of multiple video streams acquired in real time. This limits the benefits of MVI based systems for most applications.

We propose to design and develop a prototype system aimed as a platform for Multi View Imaging (MVI) applications such as Immersive Television and Video Conferencing. The project will result in a complete embedded hardware system, capable of generating real time panoramic mosaics, using video input from multiple cameras. The MVI platform will also provide basic user interaction capabilities to customize the generated panoramic video, which can assist in solving another problem in immersive TV content generation, namely the Free Viewpoint TV (FTV in Immersive TV terminology).

Since, the prototype is being developed on a reconfigurable architecture, the platform can then be used to prototype other deliverable products which are useful for the local market and export. These applications encompass Internet (Panoramic video conferencing, Tele-monitoring of an object), Media (User-defined view for sporting events, Time freeze-videos), Security (Advanced surveillance system allowing to view and compress video from multiple cameras in a single view, Real time panoramic views for armed personal vehicles). The reconfigurable MVI platform will also provide a test-bed for the academia for rapid development of video processing based research applications.


The Team

Designation

Name

Project Director

Rehan Hafiz

Joint Project Director

Mutahir Latif

Team Lead Hardware

Haris Anis

Team Lead Software

Imran Zafar

Professional Developer (H/W)

Raheel Bari

Professional Developer(S/W)

Atif Ahmed

Professional Developer(S/W)

Ashar Rasul

Professional Developer (H/W)

Saad Shoaib

Professional Developer (H/W)

Rehan Ahmed

MS (RA)

Hassaan Saadat


HR Hierarchy Tree & Responsibilities


Project Milestones and Deliverables:

NO

Elapsed time from start

Milestone

Deliverables

Status

1

3rd Month

Hiring of team resources, equipment purchase, Literature

review and software simulation

Completed.

 

 

 

Software simulation of algorithm for panoramic view and Report on the following topics.

1. Details of the Team resources hired

2. List of equipment purchased and setup for

development

3. A Literature review of existing image mosaicing algorithms

4. Matlab implementation of at least two algorithms for image mosaicing.

5. Source code

 HR Hiring : Done

S/W Simulations : Done

2

6th Month

Performance Evaluation completed. Hardware Design Specification completed.

Hardware specification ready and Report on the following topics:

1. Performance evaluation in software and evaluation of computational complexity to finalize the algorithm to be implemented in the

hardware.

2. The evaluation will cover the memory requirements, computational complexity, proposed hardware platform for realization and various hardware interfaces.

3. Hardware Design specification document for

the selected algorithm, which will include the data flow charts for implementation.

 Done

3

9th Month

RTL coding and Board Design Complete.

HDL implementation of Algorithms and Report on the following topics.

1. Complete Verilog source code along with Micro Architecture specification

2. Schematic and PCB design.

&

Information Seminar in which the detailed Hardware implementation of Algorithm and Board Design will be explained to seminar attendees

 Done

4

12th Month

Verification of RTL in simulation completed. Board and parts ordered for manufacturing.

Verification of algorithms implemented in HDL

Report on the following topics

1. Bill of Materials for board components.

2. For the purchase of cameras, an updated market survey will be performed to select the

best camera that satisfies technical requirements and gives best price performance

ratio.

3. Verification plan.

4. RTL verification results

5. Details of the tested and verified board.

 Done

5

15th Month

Board upbringing and new revision ordered if required. Utilities for automated testing of

board completed.

MVI platform board upbringing completed

And Report on the following topics

1. Board bring up methodology.

2. Low level APIs for board bring up.

3. Source code of Board Support Packages

Board Upbringing (Pending)

Board bring up methodology y(DONE)

Low level APIs for board bring up (DONE)

Source code of Board Support Packages (DONE)

6

18th Month

Revision 2 of the board completed. Verification of RTL and firmware on board completed. System software development completed.

Functional and verified MVI platform board ready. Complete source code for system

Software and Report on the following topics

1. RTL and Firmware test results.

2. Initial performance bench marks.

3. Source code of system software.

 Pending

7

20th Month

Hardware-Software integration completed. Application software

completed. Final documentation

completed.

Demonstration of the final product and Final Report And Information Seminar in which the complete design and testing cycle for the project will be explained to the seminar attendees.

In Progress

 

 

Publication

Rana Muhammad Bilal, Rehan Hafiz, Muhammad Shafique, Saad Shoaib, Asim Munawar, Jörg HenkelISOMER: Integrated Selection, Partitioning and Placement Methodology for Reconfigurable Architecturesin: IEEE/ACM International Conference on  Computer-Aided Design (ICCAD'13), San Jose, CA, USA, pp. 755-762.November 18-21, 201
Comments