Edited by Anuj Vyas.
Mixed Reality, i.e. Virtual & Augmented Reality, has been brought into strong focus now because of the COVID pandemic – with factors like travel restrictions, frequent home-office & remote trainings becoming everyday buzzwords. Our panelists, Carolin Pischl & Josef Wolfartsberger, present their experience with Mixed Reality and share key features, to help understand better the potential and current limitations in this field.
Department of Assembly Planning, BMW Group
There is an ever-present need to optimise assembly line trainings at BMW, and the following factors play a key role –
- Increasing globalisation; there are currently workers from over 20 different nations at the Munich plant itself. Hence, language barriers and tackling different cultural backgrounds are a daily business at the Munich plant.
- Assembly factories involve stressful working conditions; everything is about time. So, the quicker the training process of a worker, the better it is for productivity.
The need to continually optimise assembly line trainings led to BMW to test Mixed Reality (MR) training, as part of a blended training scenario, at their Plant Munich.
To test the viability and effectiveness of using MR in Assembly line training, BMW recently conducted a 2-day study to compare the use of Augmented Reality Head-mounted Displays (HMD) versus traditional Face-to-Face (F-2-F) trainings. The setup was as follows –
Group 1 (Traditional F-2-F Training) – 15 minutes each with trainer followed by 45 minutes on the assembly line.
Group 2 (HMD Training) – 15 minutes each on the HoloLens followed by 45 minutes on the assembly line.
Total no. of participants was 36.
The HoloLens training application was devised to mirror the traditional 4-step procedure that exists for assembly training at BMW, i.e. Preparation, Introduction, Execution and Controlling.
For the HMD training test, this was interpreted to run as follows –
How it Works:
The training application on the HoloLens used cues and markers imposed on the environment to orient the trainees with their environment and the locations of parts and tools in the factory. For the assembly line module, an overlay was applied over the engines to guide the trainees in placing the parts in the correct sequence and location.
- The time taken for the completion of tasks between trainees trained by the trainer and HMD was nearly identical.
- The overall quality was higher when training with a trainer. This was affected not only by pure training factors, but also by some technical and process instability issues in the HMD application itself.
- Trainees using HMD support made 10% less picking mistakes, 5% less assembly order mistakes and caused 60% less re-work.
- Instead of the traditional use of 3 trainers per training group, the use of HMDs enabled BMW to use just 2 trainers per group, since the AR setup needed only 1 trainer to execute it.
- Any survey results conducted into the use of new technologies like AR cannot be completely objective, since results vary based on numerous factors like personal preference, each trainees’ motivation to use new technologies, or the F-2-F trainer being more comfortable with the traditional processes instead of HMD training.
- In its current state, MR training cannot be deployed on its own; it must be setup as part of a blended scenario or mixed learning systems.
Overall, the use of MR technology was encouraging for BMW Group. BMW has currently implemented the use of HMD training in assembly at a global scale, incorporating a host of updates and upgrades based on results from the first test-run.
Centre of Excellence for Smart Production, FHOÖ Campus Steyr
AR Supported Remote Assistance Application has been developed at University of Applied Sciences, Upper Austria, in partnership with 4 companies. This Augmented Reality solution is open-source and will be available free to use at GitHub once the research program is complete. It's currently a work in progress.
Reasons for Development:
- Though there are similar existing commercial solutions that are already available, AR Supported Remote Assistance Application aims to give companies the opportunity to test MR applications for field deployment, without having to first buy expensive licenses.
- The primary objective for the university was to build an experimental testbed on which to identify the benefits and weaknesses of MR remote support in typical industrial maintenance processes.
MR applications have gained increased traction since the onset of the COVID pandemic. There is an increased demand for tools like these to reduce the need to be physically present on-location or travel half-way across the world just for repairing a tiny part on an industrial machine.
How it Works:
An operator wears smart glasses or holds a smartphone and shares his view with a remote expert who can then use annotations like pointers, arrows and crosshairs to guide the operator through the task. The software tries to find feature points in the environment to automatically stick these annotations to.
- Unity3D was used as application development environment;
- AR Foundation was used for AR elements;
- WebRTC established the communication between devices with a strong focus on security.
This setup was tested on android smartphones, tablets and some smart-glasses.
The task to be performed was to repair an industry computer; identifying the model type, removing cover using a set of screws, change a specific part i.e. a heat pipe, and finally close the computer again. The time taken for task to be completed and error rate was measured.
A real-world test was in the works with the university’s industry partners, but could not be run due to the current COVID-related circumstances. Instead, a preliminary evaluation was conducted with 30 students in laboratory settings; divided into 2 groups, Group 1 follows instructions using baseline paper instructions, Group 2 uses the Augmented Reality application on smartphones, while being guided by an expert sitting in another location.
- The time taken to complete the task was nearly identical.
- Error rate – 53% of students in the paper instruction-based group made errors, in stark contrast to the Mixed Reality remote assistance group, who had an error rate of 13%. This demonstrates that in the long run, fewer errors made will be the primary money-saving area for large companies.
- Participants suggested that simple hint popping-up or an arrow at the right location would have been sufficient, and anchoring and remote assistance features were not necessary.
- Mobile application especially on android drained the battery a lot, affecting portability.
- Also, the smart glasses today have strong restrictions on field-of-view, operating time and portability, all of these proving further that use in outdoor situations like tunnelling systems and construction sites is impractical currently. This is also the reason that the University’s industry partners are using this remote support prototype application with smartphones or tablets, instead of smart glasses.
- Tracking isn’t robust enough to counter for changer in lighting conditions.
- Usability largely depends on the use-case; is not a versatile means of training/guidance yet.
In conclusion, the results from both use-cases demonstrate similar findings - on one hand, MR technology is clearly a boon for training scenarios, reducing error rates and hugely creating time resources for big corporations, opening up real world environment for training and ably complementing human training. On the other hand, there is much room for improvement, i.e. portability and outdoor usage, practicality and over-engineering and a lack of versatility. What is unanimously agreed on by both panelists though is that Mixed Reality solutions are the present and future of corporate training.
Visit Learn Tomorrow's Events page to register for our upcoming webinars.
Anuj Vyas is a member of Learn Tomorrow. cBook.AI aims to create a new learning experience providing a personal Learning Feed, which smartly selects learning content on the basis of Key Performance Indicators (KPIs) and Artificial Intelligence.
Learn Tomorrow is an eLearning technology provider. cBook is an integrated Learning Experience Platform (LXP) aiming to create digital learning experiences with impact. Tailored to the needs of businesses, academies & trainers.