Student Assistant / Intern – Real-time/Online Neural Radiance Field Training of 3D Reconstruction Pipeline in Erlangen bei Fraunhofer Institut für Integrierte Schaltungen IIS

Neural 3D reconstruction is a growing field in computer vision focused on generating 3D representations from 2D images or videos. Learning algorithms, such as Neural Radiance Fields (NeRFs), play a significant role in this approach. NeRFs are neural networks that model the 3D shape and color of objects in a continuous 3D space. Through training on multiple 2D images, NeRFs can predict radiance at any 3D point, enabling the creation of realistic and detailed 3D models.

Currently there are limited applications of NeRFs with real-time scanning. There are recent works such as Nerfcapture and Nerf_Bridge that attempt to address it. These scanning apps include their own set of limitations. We want to evaluate these methods, build up on their implementations and eliminate their limitations to develop a scanning system that performs the following steps in incremental/online fashion:
Capture images -> camera calibration -> stream data to NeRF training pipeline -> monitor performance

For more information about our 3D reconstruction research, please refer to
https://www.iis.fraunhofer.de/en/profil/zukunftsinitiativen/artificial-intelligence/dsai.html

Piqued your interest? Then have a look at our offer!

• You evaluate state-of-the-art real-time scanning approaches proposed in e. g. Nerfcapture, Nerf_Bridge etc. and understand their challenges.
• You build up on an existing app or develop our own app to address their limitations consequently.
• You learn and implement NeRF training with streaming data.
• You acquire expertise in best practices for real-time scanning, capturing, and 3D reconstruction techniques, fostering a deep understanding of the subject matter.
• You document thoroughly all captured datasets, reconstructed models, and experiments conducted during the project.

• You are currently studying media production, electronics engineering, computer science, information and communication technologies, or a related field
• You have experience in programming using languages such as Python, C++, Java etc.
• You are interested in building a streaming pipeline based on Data Distribution Systems (DDS)

Optional:

• You know basics of image processing and 3D computer vision e.g. camera calibration, multi-view geometry etc.
• You are familiar with tools like Git, IDE (e.g. VSCode), etc.

• An interesting application-oriented field of research with innovative projects and state-of-the-art laboratory environment
• Extensive professional support from scientific mentors
• Flexible working hours that allow you to balance your studies and job experience
• An open and friendly work environment
• Sufficient opportunity to develop your interests and skills

Weekly working hours are determined by agreement. You can start from now on (as a student assistant from 10 to 20 hours a week or as an intern for a period of at least three months). You can reduce your hours before exams and increase them during semester breaks. You can flexibly determine the working days. After your studies, you have the option of working with us full or part time.

We would be happy to offer you the opportunity to write a bachelor's thesis (for a period of six months) in cooperation with us in the above-mentioned subject area. The thesis will be assigned and carried out in accordance with the rules of your university. For this reason, please discuss the thesis with a professor who can advise you over the course of the project.

If you have any questions about this job offer, please contact:
Ashutosh Mishra (ashutosh.mishra@iis.fraunhofer.de), phone: +49 9131 / 776-6789

We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity.

Apply online now (PDF: cover letter, CV, transcripts). We look forward to getting to know you!

Fraunhofer-Institute for Integrated Circuits IIS
www.iis.fraunhofer.de/en