How to discover new objects in images? Help the robot to identify when it encounters something it does not know yet! Learn to discover by a multimodal model of images and depth using deep learning.
What will be your role?
In an open world, a robot will encounter new objects. The key is to identify when an object is unknown, such that an operator can label it, or to defer to zero-shot techniques. Novelty detection in images is a well-known, hard problem. Especially for object detection, because that requires a sense of what a new object may look like, in order to localize it. Our hypothesis is that it is more feasible to detect novelties in depth images. Typically new objects have shapes that are different from the known objects, which provides a discriminative cue of novelty. And, segmentation of shapes is more straightforward than segmenting cluttered images. Therefore, shape information from the depth image (D), is a more direct cue of a novelty. After identifying the novel object, the robot may move around the object and take a few image shots of the object from different angles (RGB). The human operator only needs to provide 1 label, effectively labeling the few shots at once. From few-shot learning, it is known that a few labeled shots of an object may provide an initial model of the new object, such that it can be recognized a next time. The goal of this research is to identify novel objects in depth (D) images, to take various images (RGB) from a novel object, and to ask for one label in order to learn an initial model on these images.
What we expect from you
You are in the final stages of your degree in artificial intelligence, computer science, physics, mathematics, electrical engineering, systems and control engineering, or a similar degree and have some track record in the field of computer vision. You have experience in deep learning, python and statistics, preferably also knowledge of object detection and outlier detection. You are pragmatic and focused on making things work. Besides technical skills we value teamwork and communication skills. For this vacancy it is required that the AIVD issues a security clearance after conducting a security screening.
What you'll get in return
You want to work on the precursor of your career; a work placement gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Naturally, we provide suitable work placement compensation.
TNO as an employer
TNO is an independent research organisation whose expertise and research make an important contribution to the competitiveness of companies and organisations, to the economy and to the quality of society as a whole. Innovation with purpose is what TNO stands for. With 3000 people we develop knowledge not for its own sake but for practical application. To create new products that make life more pleasant and valuable and help companies innovate. To find creative answers to the questions posed by society. We work for a variety of customers: governments, companies, service providers and non-governmental organisations.
TNO’s Intelligent Imaging group is specialised in image processing, image enhancement, image analysis, visual pattern recognition, artificial intelligence and deep learning. This young, passionate, creative and committed group (40 people) works with partners on technological breakthroughs for innovations in important societal and economic themes.
The selection process
For this vacancy it is required that the AIVD issues a security clearance after conducting a security screening. Please visit for more information the AIVD website.
Has this job opening sparked your interest?
Then please feel free to apply on this vacancy by pressing the button on the website! For further questions don’t hesitate to contact us.
Due to Covid-19 and the consequent uncertainties and restrictions, students who are not residing in the Netherlands may currently not be able to start an internship or graduation project at TNO.
Contact: Pieter Piscaer
Phone number: firstname.lastname@example.org