Imerso makes reality capture fun again

Introduction

I have looked forward to writing this review for a long time now. I have followed the developers of Imerso from the very start and have seen their determination to break through challenges on their way. They have now reached a milestone and starts selling their software and their vision to architects, contractors and building consultants. It’s the start of an exciting new chapter for them and I’m very happy to share their work with you.

So be warned, I cannot be entirely objective on this review. I have tested their product for more than a year and have been giving the team my advice and feedback along their way. I want to share my impressions with you nonetheless, as I believe you will not want to miss this.

Why should you care about Imerso?

We are seeing a rapid change within technology in almost any professions, and luckily architecture and construction is no exception. I believe that we are on the verge of a big shift in the way we will be working with in the coming years, and that this will impact us one way or another before we know it.

As I talked briefly in my previous post, the need to get new tools for working with reality capture is vital to fully harvest its benefits. Traditional reality capture has relied on laser equipment or photogrammetry methods, which is becoming evermore available, but in general neither of these has been something architects had the knowledge or equipment to do themselves in their daily work. That is a huge lost opportunity, since we as architects rely on reality to create our designs, and so we need efficient solutions for capturing reality first.

What I want and need as an architect is an affordable, easy-to-use, accurate, and fast way of capturing reality. I want to supplement my control measurements and on-site photos with 3D field-data, so that I can sketch design layouts rapidly and efficiently, while preserving the real-life context and visuals over my models. I want to be able to document on-site work, as a project progresses, without having to take thousands of photos or manual checks

That’s a lot to ask for. To make this work, the success criteria is brutal. A solution must be intuitive to use, relatively cheap, have decent accuracy, needs to work fast, and be robust to all kinds of users and on-site conditions.

Introducing Imerso

Imerso is a 3D scanning software for mobile devices built by a young computer-vision team based in Oslo, Norway. In this post, I’ll be describing two factors of their work: the software they built and hardware you need to use it.

Hardware -Google Tango

For using Imerso, you must have a phone or a tablet that supports the Google Tango technology. This augmented reality platform was introduced by Google in 2014 and launched commercially earlier this year.It is basically a normal android device with additional kinds of embedded sensors and computing power, that lets software developers create new types of applications.

google_tango_1

There are currently two devices available with this technology, the Lenovo Phab2PRO and the Asus ZenFoneAR (launching in summer). I have been using the Lenovo Phab2PRO (ca. $400) during my testing of Imerso. So, the technology is obviously not supported by Apple and that may be a turn-off for many architects. But that should not be a show-stopper from starting to use Imerso and improve your company’s work. Imerso’s team will help you get the devices you need to get started in days, and can also send you one as part of your subscription, for your team to use.

 

Software -Imerso App

Given this minimal investment in equipment, it is the scanning software and the workflow around it that makes Imerso want to improve. Their app is intuitive and quick to get the hang of. The team is constantly improving the interface and functionality without making it too complex or making to many compromises. That is a fine balance to keep, but in my opinion they have found a pretty sweet spot.

 

 

 

 

 

Software -Web plattform

After making your scans, you upload them from your phone into Imerso’s web platform. You can access your data from anywhere using a browser, to view the results in the online viewer and quickly take measurements inside the scan. From here you can export the the 3D model to a pointcloud and/or as a mesh that you could open in your preferable design-software.

 

 

 

 

 

 

How does it work

Scanning

Let’s say you want to scan a room or apartment for designing elements over it on Revit. You do that by launching Imerso’s app and pointing your phone to the walls/ceiling/floor that you want to capture, just like you are filming them. You press the “Start” button to begin scanning, and the app shows you the 3D reconstruction being built in real-time as you move through that space. All you have to do is “fill the blanks” as you move and build up your model.

It is hard to explain this part, but it is intuitive when you try it. It’s very fun and easy to use, but having said that, there are definitely some tricks and tips to ensure great results and that comes with practice.

Here are some of the tips I’ve learned to have a great scanning results with Imerso:

  • Avoid too sudden movements. Imerso let’s you move rather fast while scanning, but you should still try to move your phone smoothly and not shake it too much. It also helps if you consider your trajectory and what you want to capture before starting to scan.
  • Don’t run the optimization tool right after saving each scan. The app does not let you make new scans while the optimization is running on another scan. Therefore, my workflow is to first make all the scans I need, and then run the optimization later on all the scans at once.
  • The real-life coloring of the scan is generated by the camera of the phone. Therefore you should avoid covering the sensors with your hand, or scanning directly against strong lights or very dark spaces since this will affect the results. If you see that parts of the mesh are getting different exposures during scan, you can simply point the device at that area again while scanning to get better color. Either way, when you optimize your scan afterwards it will look much better than what you see during real-time scanning.
  • I usually scan in portrait mode, but you should avoid hitting the physical power button on the side. This is more Lenovo’s design, than Imerso, but it is easy to accidentally hit the power button and lock the screen, forcing you to re-start your scan.
  • Be sure that you have enough storage space before you start to scan. Imerso will give you a warning if your storage is getting full, but to avoid frustration when you are on-site, be sure to have available storage by uploading scans to the webapp and/or deleting old scans from the app
  • For capturing a large area or an entire apartment, I break it down into sections and scan each part at a time. To make it easy to puzzle the pieces together later in Revit
    Using Imerso shouldn’t keep you from taking photos and control measurements. It is often a good idea to bring with you a laser pointer and take a few control measurements to check tolerances and do fine adjustments in your designs over the scans.

Limitations

Of course, there are limitations with this device. It is just a regular phone with some extra sensors. So if you compare this phone with a traditional scanner, you are comparing apples and bananas.

The most obvious limitation is the relation between accuracy and the size of the scan. The larger the area you scan in one go, the more inaccuracies you will accumulate in your scan. You can still make large scans for the purpose of context, visuals, and FDV documentation, but Imerso will give you a notification to stop scanning at about the 50-80m2 size, after which you can start another scan and keep going.

In my experience, comparing control measurements against measurements within Imerso’s scans has given me a +-1% accuracy. This means that for scans of areas within the recommended size for a single scan, the precision is quite good to work with. If you capture areas beyond this size in a single scan, measuring smaller things will still give you an accurate result, but measuring longer distances will results in less accurate measure. You should therefore take this into consideration when modeling over the scans in your designs. As I got used to the app, I became more and more comfortable with this and learned what to expect according to the scans I make.

Nonetheless, when the area you want to scan is quite big, you should break it down into sections and make separate scans. You can puzzle them together later in Revit. I also take control measurements with the laser pointer to do a quality control, which helps me do fine adjustments when putting scans together, particularly of areas where I care most for accuracy.

In addition, you have the limitation of the scanning distance from the phone. It is about 4 meters, but it depends on the surface/material that you are scanning.

The Imerso’s app is primarily designed for indoors use.  You can also scan outdoors, but too much direct sunlight will flood the infra-red sensors and affect the quality of your scan. Shadowed places, early/late hours, and cloudy days are perfect for outdoors scanning.

Functionality that is on the wish list is an “append” function. What I mean is that I could do multiple different scans, and they align correctly to each other. That was a neat functionality with the DPI-8 two years ago. The Imerso team is about to implement this functionality as we speak so stay tuned.

 

Wrap up

Exactly two years ago I wrote about the handheld scanner DPI-8. With the DPI-8 I got the feeling that the device was still in beta stage, but my experience with the scanner was still an eye-opener. I suddenly got an understanding how this technology will affect the way we will work in the future.

Then I tried the Faro Freestyle scanner half a year later. I saw then that the product segment had matured since it had better details, accuracy and tracking. But that was only true if the scan was less than 8m3. If you exceeded this, it became more inaccurate. In some cases, that is OK, but with a price-tag then at 120.000 NOK for the X version it was not that tempting. That was one and a half year ago.

And now technology like Google Tango and others, and products like Imerso and Matterport, have drastically started to lower the entrance level both regarding price and complexity. That is indeed promising.

My vision is that architects should manage to capture reality themselves when they are on the job-site, but the success criteria for this, as I have mentioned is quite high. Depending on the project, Imerso can make a huge difference in your work.

In my opinion, these technologies open a lot of opportunities to be more efficient, deliver better service, and keep your projects on track. There is a huge potential to start using these new methods early and build expertise, as they will definitely become an intrinsic part of our industry.

After all, we are living in reality and building on it. Let’s start using reality in our digital work as well.

 

Time for a «reality» check with Revit

As an architect I want my projects to be conscious to its surroundings. Sometimes I want the project to be a contrast to its surroundings, and sometimes I want the project to blend in with its surroundings. It all depends on where, what and why.

That is why I usually start creating the geometry that will impact the design of the project from the very start of a project. And instead of creating simplified volumes/masses of buildings around the project, I now want to use the geometry I have created with Reality Capture with either photogrammetry or scanning.

The reason is obvious. I get much more richness and accuracy than I get from a simple volume model. The only question now is how I can use the Reality Capture model in my design?

billingstad

A typical reality capture model that I want to use in Revit

Since there is no good way of importing a large mesh with textures into Revit, pointcloud is the only gateway for Revit users to interact with the «Reality» while designing. And the limitation of only being able to import pointcloud is not necessary a bad thing, but it then requires that Revit supports the features that make pointcloud useful in its design.

recap

Pointcloud in Recap

First of all I have to say that I am impressed on how Revit deals with huge pointcloud files. The performance working with pointcloud is surprisingly fast , and the visibility of the pointclouds when using clipping plane or section box is very solid. In addition you have the possibilities to override the colors and turn on and off «scan regions» and «scan locations». This can be very useful in many occasions.

But that is pretty much what you can do with «out of the box» Revit and pointcloud. Yes you can buy add-ins that vastly improves the functionality, but I do think Revit should step up the game and start giving its users more tools.

But ok. The tools are a bit limited but in most cases it is sufficient. Let us now focus on the benefits you have in front of you:

  • you can design your project in «reality». You see immediate how the project responds to its surroundings
  • In my experience, the municipalitiy gives positive feedback on a project when they see that the project has «true» surroundings
  • you can create an accurate and updated terrain based on a «fresh» pointcloud
  • it is easy to accurate model existing buildings, plants, trees etc.
revit

Pointcloud in Revit

Ideally you should not need to model the things that is already in the pointcloud since it is already there. But in my experience it is hard to avoid since pointclouds:

  • do not cast shadows
  • do not render in raytrace
  • do have a hard time get into VR
  • have to use third parts applications to categorize the points (wall, floor, roof, site elements etc)
  • it is not often I am presenting the project to a client within the interface of Revit. Lately I have started using Live with great success. Guess what, pointclouds will not be exported to Live. On the other hand you can import mesh into Live but good luck trying getting it correctly placed
  • change their looks depending on the printer quality settings (see photos below that illustrates this). You will have to comprise between quality of the pointcloud and Revit elements.

I understand that Revit is not supposed to solve everything and I do think that this is a good strategy. But I do believe that Revit should do some serious work done to get the pointcloud even more useful for specially presenting the project.

After all, we want the design to be a reality.

Want to work together?

As the render quality within Revit is getting better and better, in combination with less and less time for preparing presentation and deliveries, I tend to do more and more of my work within Revit.

When I need better renderings than I manage to do in Revit myself, I export the Revit project to someone that can do the job in 3D Studio with V-Ray for me. And it works just fine and I am very happy with the results.  But usually I am developing the project almost to the last hour before the deadline, and in these situations it is difficult to outsource the renderings since they need the model in advance.

revit_viz

Exporting the Revitproject for rendering in 3D Studio

This is something I want to change. Since the render quality within Revit with Raytrace or V-Ray for Revit is getting better, I want people doing the renderings for me to do it within my Revit project. In this way, I can have a value of the materials that are adjusted, the light settings that are created, the surrounding that is created, throughout the entire process. So when I export my project to Autodesk Live or other platforms, I get these settings exported straight away without much work since they already are applied.

revit

Do the work within Revit

So if you have great Revit skills, and superior skills with Raytrace or V-ray for Revit with a great aesthetic touch and totally trustful, I want to work with you on my projects. I am willing to invite you into my Revit project with Collaboration for Revit -The Holy Grail

key

If you feel that you are the ONE, please do not hesitate to give me hands up by sending me an email. It would be great to work together with you.

 

Architecture Biennale 2016

I just came back from a fabulous trip to Venice. It was my first time to the picturesque but touristic city, but my main attraction was the Venice Architecture Biennale. The Architectural Biennale was a great experience and it surely refueled my architectural interest.

Regarding digital design and architecture, I became immensely impressed by both the exhibition Beyond Bending by Block Research Group and  DeJong & Block, and the exhibition made by Forensic Architecture.

I was not aware of the work of Forensic Architecture and the way they managed to present their work was really inspiring.

 

VR and graphics cards

We have started flirting with VR at our office and when I tested if our computers were VR-ready with «SteamVR Performance Test», I was pretty surprised.

We are using Lenovo ThinkStations P900 Workstations. They are considered as high end computers with «extreme performance». These P900 have either Nivida Quadro K4200  or Nivida GeForce GTX 970 installed, but the rest of the specs are the same. You can see the results from the performance test below:

Yes, I knew that the GTX970 was superior before I tested, but I must admit that I was surprised that the K4200 did not score better.

Btw, if you are going to buy a new workstation in the near future, you should wait for the GTX 1080.

Memento Mori -reborne to Remake

It has been a joyful ride to follow the development of Memento the last couple of years. They have made photogrammetry intuitive, and changed the way my colleges and me have been working as architects.

I am happy to see products that has been in Beta, released as commercial product when they are matured and ready. For Memento, I understand that this will happen i May. Last Friday they released the last Beta version of Memento and the announced that the official product name will be Autodesk Remake.

They released couple of new features in the last Beta and you can read about them in detail here. One of the features that has been wanted for a long time is the «Offline reconstruction». This feature will enable you to process your images on your local computer instead of sending it to the Autodesk servers. This process is computationally very intensive and from the recommended specs that you find here, you may want to upgrade your computer…

I also feel that an epoch has ended. An epoch of an unlimited playground of testing and playing that was free and noncommittal. That is something that is a rare experience in our digital world and has made Memento, in addition to its intuitive tools, unique.

I am looking forward seeing the final commercial release and I will follow Memento closely in the future. I hope the software team manage to keep Autodesk Remake as intuitive and uncompromising as possible, and that they not are trying to solve too many problems in one software.

Phantom 4 -love at first sight

Three weeks ago, I had never flown a real drone before. However, since my appetite for reality capture has been growing lately, I felt it was time to invest in a drone that could be used for photogrammetry. Therefore, after comparing/evaluating/comparing again different types of drones, I finally decided to buy the Phantom 4.

I am always afraid of being disappointed when I finally receive something that I have been waiting for a long time, but in this case, it was no need to worry. DJI made even the unboxing an Apple-like feeling, and when the drone was revealed I was stunned by the sexy and elegant design. I was so starstruck that it was the first time in my life that I read the manual, saw all the instruction videos on YouTube, before I even turned it on. The Phantom 4 became «my precious».

When I finally had the guts to try it outdoor, it was a joyful experience. The software was intuitive, the controller felt solid and precise, and the drone was responsive. I have tested out most of the features like the «activetrack», «tapfly», «obstacle sensing system», «return to home» and many more. I do start to trust these features, but it still feels crazy to fly the drone with full speed towards a wall and trust the «obstacle sensing system» to automatically stop your drone before it crashes.

I have also tested the Pix4D app with the Phantom 4 two times already. It is a great app and I will review it in a later blogpost. I generated a Sketchfab model from my latest flight. You can see the result below:

At the moment I am not using «my precious» for commercial use. I want to be a better pilot first, and I need to apply to the Civil Aviation Authority to get the right approval. This is important to me, since I see the potential danger a drone can cause.

 

RTC Budapest 2015 -this time as a speaker

RTC Europe (Revit Technology Conference) is an annual educational and networking event for BIM users worldwide. This is the place to be for top notch sessions, sharing of knowledge and catching up on the latest trends within digital tools. I have been to every RTC event since it started here in Europa and this time Budapest was the luck city to host this event.

This year, I was not only a delegate at the RTC Europe in Budapest, but I was also a speaker. Together with  Margarida Barbosa from Beck group we delivered a session on «Using Photogrammetric Surveys at Architectural Offices» where we explained the potential of using photogrammetry in the daily workflow.

By combining the eminent knowledge Margarida has on this subject, with my experience from work, we delivered a 75 min session for «all levels». It was a joyful experience and we were super proud to be rated no. 5 top speaker at the conference. To see the entire top ten list, check out the blog posted by Jose Fandos.

Do not hesitate to contact us if you have any questions related to this subject. We are happy to share our discoveries.

See also the review by Jack White, BIMcrunch Editorial – RTC Europe 2015 Review

 

See you in Portugal next year!

 

I have a video but all I want is a beautiful 3D model

Instead of taking multiple photos of an object to create a 3D model, you should consider capturing video of the object instead. As we all know, a video consists of multiple frames that are stacked together and the frame rate is somewhere between 24 and 30. So basically you can just extract the frames from the video and use these frames to create a 3D model by using photogrammetry.

This is one way you can do it:

  • open your video in Photoshop
  • press the «Render Video» button in your «timeline» tab
  • select the «Photoshop Image Sequence» and be sure you have maximum quality for the JPG´s
  • if you are going to use Memento, you can not exceed 250 photos. I usually divide 250 with the duration of the video in seconds. In this way, I will not extract more than 250 frames
  • hit the «Render» button and wait

You have now approximately 250 photos waiting for you to import into Memento and then it is just to wait for the processing to be completed.

Voilà!

PS! I have all my personal videos stored in «Google Photos». Have Google ever considered starting making 3D model effects from these videos by using photogrammetry, or are they just waiting for «Google Tango» to make the first dance…?

Case study #3 Part C: The final result

It seems like a decade since I wrote my first post about the modular facade:

But now the facade has been be fabricated, transported and mounted, and it is time to judge the final results.

Let us start with the construction phase. I am not found of surprises and that is why I detailed the facade down to the nuts and bolts. I was concerned that the manufacturer was not able to deliver good solution for important detail/concepts and therefor I wanted to be proactive. The result was that there were no bad surprises making the modules or when mounting them to the building.

And frankly, I am quite happy with the facade. I like how the facade reflects the light on the aluminium cladding, and the champagne color is playing well with the surrounding buildings. And keep in mind it is not an expensive facade, but together with the manufacturer we have found good solutions that we have discussed thoroughly.