Zuckerberg: Apple, Meta are in “deep, philosophical competition”

Meta CEO Mark Zuckerberg.

Meta CEO Mark Zuckerberg. (credit: Facebook)

Meta and Apple are entering a period of “very deep, philosophical competition” that will define the future of the Internet, according to comments by Meta CEO Mark Zuckerberg obtained by The Verge.

Both Apple and Meta are planning to invest heavily in mixed reality over the next decade, but they have diametrically opposed visions for what the AR/VR/XR landscape should ideally look like.

The Verge obtained an audio recording of an all-hands employee meeting at Meta, in which Zuckerberg answered an employee question about the company’s future competition with Apple in great detail. His comments shed some light on how Meta, at least, sees the rivalry.

Read 8 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #mark-zuckerberg, #meta, #metaverse, #mixed-reality, #tech, #tim-cook, #virtual-reality, #vr, #xr

IDC: “All eyes will be on Apple” as Meta’s VR strategy “isn’t sustainable”

Screenshot of promotional video for VR equipment.

Enlarge / The Oculus Quest 2.

A recent media release from market research firm IDC predicts that Meta (the parent company of Facebook) may not be able to compete in the mixed-reality business in the long run if its strategy remains unchanged.

The media release offers a bird’s-eye view of the virtual reality hardware marketplace. In the release, IDC research manager Jitesh Ubrani said that, while “Meta continues to pour dollars into developing the metaverse, [the company’s] strategy of promoting low-cost hardware at the expense of profitability isn’t sustainable in the long run.”

A similar concern was raised by tech industry analyst Ming-Chi Kuo late last month. Kuo predicted that Meta would make moves to scale down investment in virtual reality, creating an opening for Apple and other competitors. He also wrote that Meta’s practice of selling VR headsets at a loss is unsustainable.

Read 4 remaining paragraphs | Comments

#apple, #ar, #bytedance, #idc, #meta, #oculus, #oculus-quest-2, #pico, #tech, #vr, #xr

Forget smart glasses, this smart contact lens prototype has a new vision for AR 

Woman putting in a contact lens

Enlarge / Smart contact lenses don’t work quite this easily yet. (credit: Getty)

Since 2015, a California-based company called Mojo Vision has been developing smart contact lenses. Like smart glasses, the idea is to put helpful AR graphics in front of your eyes to help accomplish daily tasks. Now, a functioning prototype brings us closer to seeing a final product.

In a blog post this week, Drew Perkins, the CEO of Mojo Vision, said he was the first to have an “on-eye demonstration of a feature-complete augmented reality smart contact lens.” In an interview with CNET, he said he’s been wearing only one contact at a time for hour-long durations. Eventually, Mojo Vision would like users to be able to wear two Mojo Lens simultaneously and create 3D visual overlays, the publication said.

According to his blog, the CEO could see a compass through the contact and an on-screen teleprompter with a quote written on it. He also recalled viewing a green, monochromatic image of Albert Einstein to CNET.

Read 10 remaining paragraphs | Comments

#ar, #augmented-reality, #smart-contact-lens, #smart-glasses, #tech

Apple’s AR/VR headset will arrive in January 2023, analyst projects

An early augmented reality demo by Apple, using a smartphone instead of a headset.

Enlarge / An early augmented reality demo by Apple, using a smartphone instead of a headset. (credit: Apple)

Tech industry analyst Ming-Chi Kuo has offered the most specific prediction about a release date for an Apple augmented reality/virtual reality headset yet: January 2023.

Kuo has often made accurate, informed predictions about Apple’s plans in the past, based partly on information from sources in the company’s supply chain. On Thursday, he published a lengthy analysis of the VR headset industry and predicted that Apple’s device will “likely” arrive in January.

Kuo called the headset “the most complicated product Apple has ever designed,” noting that many current Apple suppliers are involved in the supply chain for the product. He also supported other recent leaks and speculation that the upcoming headset will not be exclusively or primarily focused on augmented reality (which places virtual options in real-world space) rather than virtual reality (which immerses the wearer in an entirely virtual space).

Read 5 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #meta, #ming-chi-kuo, #mixed-reality, #tech, #tim-cook, #virtual-reality, #vr, #xr

What to expect at WWDC 2022: iOS 16, M2, and more

The image Apple shared alongside the WWDC 2022 announcement.

Enlarge / The image Apple shared alongside the WWDC 2022 announcement. (credit: Apple)

June 6 marks the beginning of Apple’s annual developer conference, WWDC. The week-long event will kick off with a keynote at 10 am PST on Monday loaded with announcements about new software features across Apple’s various platforms.

For the first time since before the COVID-19 pandemic started, there will be a significant in-person audience for WWDC, too. And there will be countless sessions during the week on programming APIs, Swift features, and so on. But for most people around the world, new operating systems and hardware announcements are the main draw—and we’ll see a few of those during the keynote on Monday.

It’s important to note that WWDC isn’t typically focused on product announcements for consumers. It’s a place where Apple introduces new technologies and tools to developers.

Read 30 remaining paragraphs | Comments

#apple, #apple-m2, #ar, #ios, #ios-16, #ipados, #ipados-16, #m2, #mac-mini, #mac-pro, #macbook-air, #macbook-pro, #macos-13, #tech, #tvos, #tvos-15, #vr, #watchos, #watchos-9, #wwdc, #wwdc-2022, #xr

The full saga of Apple’s troubled mixed reality headset has been revealed

A man in a tee-shirt sits onstage.

Enlarge / Jony Ive speaks onstage during the 2017 New Yorker TechFest in New York City. (credit: Brian Ach/Getty Images)

A series of reports in The Information paint a detailed picture of progression, politics, and problems facing Apple’s plan to develop a virtual, augmented, or mixed reality headset since the initiative picked up steam back in 2015.

Citing several people familiar with the product, including some who worked on it directly, the reports describe a contest of wills over the direction of the device. The standoff was between Apple’s mixed reality product team (called the “Technology Development Group”) and famed Apple designer Jony Ive and his industrial design team. The report sheds light on Apple’s direction for the device, which Bloomberg recently reported is nearing launch.

They also claim that Apple CEO Tim Cook has been relatively hands-off from the product compared to others like the iPhone, and that the Technology Development Group’s location in a separate office from the main Apple headquarters has been a source of problems and frustration.

Read 11 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #jonathan-ive, #jony-ive, #mike-rockwell, #mixed-reality, #reality-os, #ros, #tech, #tim-cook, #virtual-reality, #vr, #xr

Amazon is hiring to build an “advanced” and “magical” AR/VR product

The "Sword of Damocles" head-mounted display, the original augmented reality headset, circa 1968.

Enlarge / The “Sword of Damocles” head-mounted display, the original augmented reality headset, circa 1968. (credit: Ivan Sutherland)

Amazon plans to join other tech giants like Apple, Google, and Meta in building its own mass-market augmented reality product, job listings discovered by Protocol suggest.

The numerous related jobs included roles in computer vision, product management, and more. They reportedly referenced “XR/AR devices” and “an advanced XR research concept.” Since Protocol ran its report on Monday, several of the job listings referenced have been taken down, and others have had specific language about products removed.

For example, Protocol wrote that the description for the role Sr. Technical Program Manager, New Products contained the phrase “you will develop an advanced XR research concept into a magical and useful new-to-world consumer product.” Now simply reads, “you will develop a magical and useful consumer product,” though it also says, “our team specializes in inventing new-to-world, category creating products using advanced sensing, display, and machine learning technologies.”

Read 5 remaining paragraphs | Comments

#amazon, #ar, #augmented-reality, #tech, #xr

Meta announces plans to monetize the Metaverse, and creators are not happy

A purchase confirmation dialog in Horizon Worlds.

Enlarge / A purchase confirmation dialog in Horizon Worlds. (credit: Meta)

Meta, the company formerly known as Facebook, announced some initial plans on Wednesday to allow content creators to monetize in its would-be Metaverse platform, Horizon Worlds. Meta’s planned revenue share for contributors’ creations could add up to nearly 50 percent.

Horizon Worlds is a network of shared 3D spaces that is currently exclusively available on Oculus Quest headsets. (Meta has plans to bring it to mobile, game consoles, and desktop VR in the coming months and years.)

There are already people creating spaces for Horizon Worlds, including a virtual yoga studio and a Second Life-like fast-food brand integration in the form of the “Wendyverse.” But to date, Horizon Worlds has not offered the tools for creators to make a living creating that content like they could on similar services like Roblox.

Read 10 remaining paragraphs | Comments

#apple, #ar, #facebook, #horizon-worlds, #mark-zuckerberg, #meta, #metaverse, #oculus, #oculus-quest, #tech, #vr

Apple’s AR/VR headset isn’t just a prototype anymore, sources say

An augmented reality demo by Apple, using a smartphone instead of a headset.

Enlarge / An augmented reality demo by Apple, using a smartphone instead of a headset. (credit: Apple)

Apple’s mixed reality headset has moved beyond the prototype phases and is barreling toward production, according to a new report in DigiTimes that cites component suppliers. DigiTimes claims that Apple has already conducted “second-phase engineering validation and testing (EVT 2)” for the headset.

“EVT 2” is a phase along Apple’s path to production. The company begins with prototypes before moving on to the first EVT (engineering validation testing) phase. “EVT 2” indicates that this is the second phase of testing for the device during that phase.

After engineering validation, Apple moves on to design validation and then to production validation before production finally begins.

Read 5 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #digitimes, #mixed-reality, #tech, #virtual-reality, #vr, #xr

Apple’s “realityOS” surfaces in GitHub commits, App Store logs

An enormous ring-shaped building on a green campus.

Enlarge / Apple’s global headquarters in Cupertino, California. (credit: Sam Hall/Bloomberg via Getty Images)

Software developers have discovered apparent references to a new Apple operating system called “realityOS” in App Store upload logs and in GitHub repositories used by the company.

The references were shared widely by developers Rens Verhoeven and Steve Troughton-Smith on Twitter. Verhoeven tweeted:

The tweet was accompanied by a screenshot from the logs that included “com.apple.platform.realityos” alongside a similar reference for an existing platform, “com.apple.platform.watchos.”

Read 7 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #github, #ios, #mark-gurman, #ming-chi-kuo, #mixed-reality, #realityos, #rens-verhoeven, #ros, #steve-troughton-smith, #tech, #virtual-reality, #vr, #xr

First details leak on Project Iris, Google’s next AR headset

Promotional image of AR glasses.

Enlarge / Product photography of the Google Glass wearable. Project Iris won’t look like this; it is said to more closely resemble ski goggles than casual glasses. (credit: Google)

Google engineers are developing a new augmented reality (AR) headset, according to a report by The Verge citing two people familiar with the project.

Google hopes to ship the product—codenamed “Project Iris”—sometime in 2024, but that date is likely not set in stone.

Like Apple’s rumored mixed reality glasses, Project Iris would be wireless and use external cameras to send an augmented image of the real world to you. And like one of the devices Apple has reportedly worked on, the glasses would leave the heavy-duty graphics processing to an external computer. In Google’s case, the device will rely on cloud computing instead of nearby hardware.

Read 7 remaining paragraphs | Comments

#ar, #augmented-reality, #clay-bavor, #google, #google-glass, #mixed-reality, #project-iris, #project-starline, #tech, #the-verge, #virtual-reality, #vr, #xr

Report: Apple’s first AR/VR headset faces delays

The "Sword of Damocles" head-mounted display, the original augmented reality headset, circa 1968.

Enlarge / The “Sword of Damocles” head-mounted display, the original augmented reality headset, circa 1968. (credit: Ivan Sutherland)

Apple may delay the launch of its first mixed reality headset, according to Bloomberg.

Multiple sources had previously claimed that the device was likely to launch in 2022, and Apple seemed poised to introduce its new mixed reality platform to developers at the company’s annual Worldwide Developers Conference (WWDC) this June.

But according to “people familiar with the situation” with whom Bloomberg reporters Mark Gurman, Takashi Mochizuki, and Debby Wu spoke, the announcement of the new headset could fall to “the end of 2022 or later, with the product hitting shelves by 2023.”

Read 7 remaining paragraphs | Comments

#apple, #apple-vision, #ar, #augmented-reality, #bloomberg, #mark-gurman, #mixed-reality, #tech, #virtual-reality, #vr, #xr

Report: Meta pulls the plug on its AR/VR operating system ambitions

The Oculus Quest 2, Meta's most popular VR headset today.

Enlarge / The Oculus Quest 2, Meta’s most popular VR headset today. (credit: Sam Machkovech)

Meta, the company formerly known as Facebook, has pulled the plug on its current efforts to develop an operating system for AR and VR devices, The Information reported today.

Citing “two people familiar with the decision,” the article claims that Meta will return to the status quo of running Oculus devices—and perhaps future mixed reality devices—on a modified version of Google’s Android operating system for mobile phones.

The project, which was internally called XROS, had reportedly been underway for years and “involved hundreds of employees.” Meta CEO Mark Zuckerberg was talking up its potential only a few short months ago. The reasons for Meta’s decision to pull the plug are not publicly known at this time.

Read 7 remaining paragraphs | Comments

#apple, #ar, #facebook, #mark-zuckerberg, #meta, #mixed-reality, #tech, #virtual-reality, #vr, #xr

What to expect from Apple in 2022: ARM desktops, portless iPhones, and more

Close-up photograph of the ports on the side of a notebook computer.

Enlarge / The 2021 MacBook Pro with MagSafe isn’t the last Apple Silicon transition we’ll see. The Mac Pro and 27-inch iMac will probably be updated in 2022. (credit: Samuel Axon)

2021 might have been the calm before the storm.

Except for the introduction of a few new Apple Silicon Macs, 2021 has been a quiet year for Apple. The new iPhones offered improved cameras and battery life but were otherwise nearly identical to 2020’s models. And apart from a slight bump in screen size, the new Apple Watch is barely distinguishable from its predecessors.

As 2021 draws to a close and we look ahead to 2022, it’s a safe bet that next year is going to be a lot more interesting. So we have some predictions to share.

Read 26 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #imac-pro, #iphone-14, #m1-max, #m1-pro, #m2, #mac-pro, #mixed-reality, #tech, #virtual-reality, #vr, #xr

Google is building a new augmented reality device and operating system

Google Glass for Enterprise. Google's job listings suggest that the new device and platform would be more mass-market than the one picture here.

Enlarge / Google Glass for Enterprise. Google’s job listings suggest that the new device and platform would be more mass-market than the one picture here. (credit: Google)

Google was one of the early leaders in the first wave of modern augmented reality (AR) research and devices, but the company has appeared to cool to AR in recent years even as Apple and Facebook have invested heavily in it. But it looks like that trend will soon be reversed.

On LinkedIn, operating system engineering director Mark Lucovsky announced that he has joined Google. He previously headed up mixed reality operating system work for Meta, and before that he was one of the key architects of Windows NT at Microsoft. “My role is to lead the Operating System team for Augmented Reality at Google,” he wrote.

He also posted a link to some job listings at Google that give the impression Google is getting just as serious about AR as Apple or Meta.

Read 7 remaining paragraphs | Comments

#ar, #augmented-reality, #google, #mixed-reality, #tech, #xr

With Mesh for Teams, Microsoft plans to bring 3D workspaces to remote workers in 2022

An interface and virtual workspace for Mesh for Teams.

Enlarge / An interface and virtual workspace for Mesh for Teams. (credit: Microsoft)

Microsoft has announced its intention to create an immersive 3D platform called “Mesh for Teams” for virtual meetings. As the name suggests, Mesh for Teams builds on the company’s existing Teams collaboration platform and implements the mixed reality features of Microsoft Mesh.

Announced earlier this year, Mesh is a platform for virtual meetings and other collaborative gatherings in mixed reality (a catch-all term for virtual reality, augmented reality, or any combination of the two) using a variety of devices like the company’s own HoloLens products and Windows Mixed Reality headsets, among others. Users would have persistent avatars that accurately reflect their body language and facial expressions and would be able to wander around a virtual workplace.

Workplaces would use Mesh for Teams to invite employees to log in to 3D or 2D collaborative workspaces. Sitting around a virtual conference table, workers would be able to do some things that aren’t possible in the real world. For example, a presenter could see her notes in 3D space near a virtual white board while those watching the presentation only see what she writes on the board.

Read 9 remaining paragraphs | Comments

#ar, #augmented-reality, #meetings, #metaverse, #microsoft, #microsoft-mesh, #microsoft-mesh-for-teams, #microsoft-teams, #mixed-reality, #tech, #virtual-reality, #virtual-workplace, #vr, #xr

Apple’s first headset will focused on “high-quality” games, reporter claims

An augmented reality demo by Apple.

Enlarge / An augmented reality demo by Apple. (credit: Apple)

In the past few days, two new reports have shed light on the specifications and strategy behind Apple’s upcoming mixed reality headset. Both claim that Apple is on a path to launching its first augmented reality/virtual reality headset as soon as next year and that the product will feature ultrahigh-end specifications and technologies.

Writing in his weekly Bloomberg newsletter, reporter Mark Gurman says the new headset will feature “advanced” chips, displays, and sensors and that it will have “avatar-based features.” That latter point indicates that Apple has a similar vision for how the headset could be used to that of Meta, whose CEO Mark Zuckerberg has been declaring a vision of social interconnectedness through AR and VR experiences.

Gurman also writes that Apple’s first headset will be a mixed reality one, supporting both VR and AR applications. The long-rumored consumer AR glasses will come much later, “years down the road.” While investment has been pouring into research on AR technology, there remain numerous major technological roadblocks to mass-market consumer AR glasses, and it is unclear when all that investment might translate into a viable, mainstream product.

Read 3 remaining paragraphs | Comments

#apple, #ar, #augmented-reality, #bloomberg, #mark-gurman, #ming-chi-kuo, #mixed-reality, #tech, #virtual-reality, #vr, #xr

Can Pikmin Bloom recapture the magic of Pokémon Go?

...and you'll neeeeeveeeeerrrr waaaaaaalk allooooooooooone.

Enlarge / …and you’ll neeeeeveeeeerrrr waaaaaaalk allooooooooooone.

The first and last time Nintendo collaborated with location-based AR company Niantic, the result was the worldwide mega-phenomenon Pokémon Go. Five years later, the companies are working together once again to see if they can recapture the magic with a new augmented reality game based on the much more niche Pikmin franchise.

After testing an early version of the Pikmin Bloom app over the last week, I can say that the game serves as an effective, super-cute pedometer, providing some nice, gentle motivation for reluctant walkers to get up and get their daily steps in. But while this gamified Fitbit requires less fuss and direct hassle than Pokémon Go, the game’s basic “watch the numbers go up” loops also don’t have the same compulsive collect-them-all appeal as Niantic’s previous hit.

Watching numbers go up

For the uninitiated, Pikmin are tiny, colorful, slightly humanoid creatures with blooming flowers on their heads. In the original console games, your character grows and manages an expanding team of Pikmin with varying abilities to help a marooned spaceman escape a planet. In Bloom, the Pikmin who follow you on your daily walks are more concerned with planting petals and growing normal flowers, which show up permanently on the game’s map.

Read 12 remaining paragraphs | Comments

#ar, #augmented-reality, #gaming-culture, #niantic, #nintendo, #pikmin, #pokemon

VR, AR, wearables, and smart home tech are now mainstream, research says

It wasn’t long ago that virtual reality (VR) and augmented reality (AR) headsets, smartwatches, and voice-controlled homes were the fantasies of books and movies. Today, VR, AR, wearables, and smart home tech have passed the early-adoption phase and are all firmly part of the “mass market,” according to research that the International Data Corporation (IDC) shared today. The global research firm predicted that the combined market will hit $369.6 billion by the end of 2021 and grow to $524.9 billion in 2025. 

IDC expects AR and VR combined to show the most growth out of the three categories, thanks to both businesses and individual consumers. The latter is particularly interested in “robust gaming solutions,” IDC said. Businesses represent the bulk of AR spending today, but IDC thinks the market for AR headsets targeting the general public will grow. It predicted a 67.9 percent compound annual growth rate from 2020 to 2025 for AR and VR combined, which is more than 10 times the next competitor, smart home tech (10.1 percent growth rate).

Smart home tech will represent the most valuable market, however, with a predicted 2025 value of over $400.3 billion. The biggest sellers will reportedly be smart TVs, streaming players, and other “networked entertainment devices,” which are expected to represent $229 billion in 2025. 

Read 3 remaining paragraphs | Comments

#ar, #augmented-reality, #idc, #smart-home, #tech, #uncategorized, #virtual-reality, #vr, #wearables

Xiaomi launches its own smart glasses, of course

Xiaomi is challenging Facebook in the wearables arena by launching its own smart glasses. The device won’t only be capable of taking photos, but also of displaying messages and notifications, making calls, providing navigation and translating text right in real time in front of your eyes. Like Facebook, Xiaomi is also putting emphasis on the device’s lightness despite its features. At 51 grams, though, it’s a bit heavier than the social network’s Ray-Ban Stories. In addition, the glasses also has an indicator light that shows when the 5-megapixel camera is in use.

Xiaomi’s Smart Glasses are powered by a quad-core ARM processor and run on Android. They also use MicroLED imaging technology, which is known for having a higher brightness and longer lifespan than OLED. The company says the technology has a simpler structure that enabled it to create a compact display with individual pixels sized at 4μm. You won’t be able to view the images you take in color, though — Xiaomi says it opted to use a monochrome display solution “to allow sufficient light to pass through complicated optical structures.”

The company explains:

“The grating structure etched onto the inner surface of the lens allows light to be refracted in a unique way, directing it safely into the human eye. The refraction process involves bouncing light beams countless times, allowing the human eye to see a complete image, and greatly increasing usability while wearing. All this is done inside a single lens, instead of using complicated multiples lens systems, mirrors, or half mirrors as some other products do.”

Its smart glasses won’t be just a second screen for your phone, according to Xiaomi. It’s independently capable of many things, such as selecting the most important notifications to show you, including smart home alarms and messages from important contacts. The device’s navigation capability can display maps and directions in front of your eyes. It can also show you the number of whoever’s currently calling your phone, and you can take the call using the smart glasses’ built in mic and speakers.

That mic will be able to pick up speech, as well, which Xiaomi’s proprietary translating algorithm can translate in real time. The glasses’ translation feature also works’ on written text and text on photos captures through its camera. Unfortunately, the company has yet to announce a price or a launch date for the glasses, but we’ll keep you updated when it does.

Editor’s note: This article originally appeared on Engadget.

#ar, #augmented-reality, #column, #gadgets, #hardware, #smart-glasses, #tc, #tceng, #xiaomi

Ray-Ban Stories: These are Facebook’s first mass-market smart glasses

As previously rumored, Facebook has partnered with EssilorLuxottica to produce Ray-Ban Stories, one of the first potentially viable attempts at mass-market smart glasses. They are similar in some ways to early iterations of Snapchat Spectacles but with a more stylish aesthetic that looks right in line with other Ray-Ban glasses.

The glasses have two front-facing cameras, each at 5 megapixels. Users can take a photo either with a touch gesture or with a “Hey Facebook” voice command. So people in the room can tell that pictures or video are being taken, a white LED on the front of the frames will light up. Videos can be as long as 30 seconds.

Photos and videos taken with the glasses are sent to a new smartphone app called Facebook View, which offers essential editing and sharing capabilities. In addition to photo and video capture, the glasses allow you to take calls or listen to music and podcasts or with built-in speakers and microphones.

Read 8 remaining paragraphs | Comments

#ar, #augmented-reality, #facebook, #ray-ban, #smart-glasses, #tech, #wearables

TikTok is building its own AR development platform, TikTok Effect Studio

Both Facebook and Snap offer tools that allow developers to build out augmented reality (AR) experiences and features for their own respective family of apps. Now, TikTok is looking to do the same. The company recently launched a new creative toolset called TikTok Effect Studio, currently in private beta testing, which will allow its own developer community to build AR effects for TikTok’s short-form video app.

On a new website titled “Effect House,” TikTok asks interested developers to sign up for early access to Effect Studio.

On the form provided, developers fill out their name, email, TikTok account info, company, and level of experience with building for AR, as well as examples of their work. The website also asks if they’re using a Mac or PC (presumably to gauge which desktop platform to prioritize), and whether they would test Effect House for work or for personal use.

The project was first spotted by social media consultant Matt Navarra, via a tip from Sam Schmir.

TikTok confirmed to TechCrunch the website launched earlier in August, but the project itself is still in the early stages of testing in only a few select markets, one of which is the U.S.

The company couldn’t offer a timeframe as to when these tools would become more broadly available. Instead, TikTok characterized Effect Studio as an early “experiment,” adding that some of its experiments don’t always make it to launch. Plus, other experiments may undergo significant changes between their early beta phases and what later becomes a public product.

That said, the launch of an AR toolset would make TikTok more competitive with industry rivals, who today rely on creative communities to expand their apps’ features sets with new features and experiences. Snap, for example, launched a $3.5 million fund last year directed toward Snapchat AR Lens creation. Meanwhile, at Facebook’s F8 developer conference in June, the company announced it had grown its Spark AR platform to over 600,000 creators across 190 countries, making it the largest mobile AR platform worldwide.

Image Credits: screenshot of TikTok website

TikTok, too, has been increasing its investment in developer tools over the past couple of years. However, its focus as of late has been on toolkits aimed at third-party developers who want to integrate more closely with TikTok in their own apps. Today, TikTok’s developer website provides access to tools that allow app makers to add TikTok features to their apps like user authentication flows, sound sharing, and others that allow users to publish videos from a third-party editing app out to TikTok.

The new TikTok Effect Studio isn’t meant to be used with third-party apps, however.

Instead, it’s about building AR experiences (and possibly, other creative effects), that would be provided to TikTok users directly in the consumer-facing video app.

Though willing to confirm its broader goals for TikTok Effect Studio, the company declined to share specific details about the exact tools may be included, citing the project’s early days.

“We’re always thinking about new ways to bring value to our community and enrich the TikTok experience,” a TikTok spokesperson told TechCrunch. “Currently, we’re experimenting with ways to give creators additional tools to bring their creative ideas to life for the TikTok community,” they added.

#apps, #ar, #augmented-reality, #creative, #creators, #developers, #facebook, #mobile-applications, #snap, #snapchat, #social-media, #software, #spark-ar, #tiktok, #united-states, #video-hosting

Apple‘s Tim Cook: Sideloading is “not in the best interests of the user”

Apple CEO Tim Cook being interviewed remotely by Brut.

Enlarge / Apple CEO Tim Cook being interviewed remotely by Brut. (credit: Brut.)

Apple has been under a mountain of scrutiny lately from legislators, developers, judges, and users. Amidst all that, CEO Tim Cook sat with publication Brut. to discuss Apple’s strategy and policies. The short but wide-ranging interview offered some insight into where Apple plans to go in the future.

As is so common when Tim Cook speaks publicly, privacy was a major focus. His response to a question about its importance was the same one we’ve heard from him many times: “we see it as a basic human right, a fundamental human right.” Noting Apple has been focused on privacy for a long time.

He explained:

Read 10 remaining paragraphs | Comments

#app-store, #apple, #apple-app-store, #apple-watch, #apps, #ar, #augmented-reality, #ios, #iphone, #privacy, #sideloading, #tech, #tim-cook

Everyone you know is a Disney princess, which means AR is queen

This weekend, all of your friends morphed one by one into animated, Pixar-inspired characters. This isn’t a fever dream, and you’re not alone.

On Thursday, Snapchat released a Cartoon 3D Style Lens, which uses AR to make you look like a background character from “Frozen.” Naturally, even though TikTok’s own AR cartoon effects aren’t quite as convincing as Snapchat’s, people are turning to TikTok to share videos of themselves as Disney princesses, because of course they are.

This isn’t the first time that a Disney-esque AR trend has gone viral. In August 2020, Snapchat had 28.5 million new installs, which was its biggest month since May 2019, when it got 41.2 million new installs. It might not be a coincidence that in early August 2020, Snapchat released the Cartoon Face lens, which users realized could be used to “Disneyfy” their pets – the tag #disneydog got 40.9 million views across platforms on TikTok. Then, Snapchat struck viral gold again in December, when they released the Cartoon lens, which rendered more realistic results for human faces than the previous iteration.

According to Sensor Tower, Snapchat’s global installs continued to climb month-over-month throughout the rest of 2020, though installs slightly declined in December. Still, Snapchat got 36 million downloads that month. Now, after the newest Cartoon Style 3D lens went viral again, Snapchat hit number 6 on the App Store’s free apps charts, compared to TikTok’s number 2 slot. Still, Snapchat downloads in May were 32 million, down from 34 million in April, while TikTok saw 80.3 million installs in May, up from 59.3 million in April.

Image Credits: Snapchat, screenshots by TechCrunch

But there’s a new app in the number 1 slot that also made an impact on this weekend’s cartoon explosion. Released in March, Voilà AI Artist is yet another platform that turns us into cartoon versions of ourselves. Unlike the AR-powered effects on Snapchat or TikTok, Voilà is a photo editor. Users upload a selfie, and after watching an ad (the ad-free version costs $3 per week), it reveals what you would look like as a cartoon.

Voilà AI Artist was only downloaded 400 times globally in March 2021. By May, the app surpassed 1 million downloads, and during the first two weeks of this month alone, the app has been downloaded over 10.5 million times.

Again, like the repetitive iterations on the “Disneyfy” trend, apps like Voilà aren’t new. FaceApp went viral in 2019, showing people what they’ll look like when they’re old, graying, and wrinkled. The app became the center of a privacy controversy, since it uploaded users’ photos to the cloud to edit their selfies with AI. FaceApp made a statement that it “might store updated photos in the cloud” for “performance and traffic reasons,” but that “most images” are deleted “within 48 hours.” Still, this ambiguous language set off the warning bells, urging us to think about the potentially nefarious implications of seeing what we’ll look like in sixty years. Two years earlier, FaceApp put out a “hotness” filter, which made users’ skin lighter – FaceApp apologized for its racist AI. Voilà, which is owned by Wemagine.AI LLP in Canada, has also been criticized for its AI’s eurocentrism. As these apps grow in popularity, they can also uphold some of our culture’s most harmful biases.

Image Credits: Voilà

Like FaceApp, Voilà requires an internet connection to use the app. Additionally, its terms outline that users grant the company “a non-exclusive, worldwide, royalty-free, sublicensable, and transferable license to host, store, use in any way, display, reproduce, modify, adapt, edit, publish, and distribute Uploaded and Generated content.” Basically, that means that if you upload an image to the platform, Voilà has the right to use it, but they don’t own it. This isn’t abnormal for these apps – when we upload photos to Instagram, for example, we also grant the platform the right to use our images.

Still, it’s a good thing that apps like Voilà force us to consider what we give up in exchange for the knowledge that we’d make a good Disney princess. Earlier this month, TikTok updated its U.S. privacy policy to dictate that the app “may collect biometric identifiers and biometric information” from users’ content. This includes “faceprints and voiceprints,” terms that TikTok left undefined. When TechCrunch reached Tiktok for comment, they couldn’t confirm why the terms now changed to allow for the automatic collection of biometric data, which refers to any features, measurements, or characteristics of our body that distinguish us, even fingerprints.

It’s no wonder that as Voilà climbed to the number one slot on the App Store, Snapchat re-upped their Pixar-inspired AR lens. Facebook’s own Spark AR platform is rolling out new features, and last week at WWDC, Apple announced a major update to RealityKit, its AR software. But these trends reveal more about our growing comfort with face-altering AR than they do about our nostalgia for Disney.

#app-store, #apple, #apps, #ar, #augmented-reality, #canada, #computing, #disney, #instagram, #internet-culture, #mobile-applications, #photo-editor, #realitykit, #snapchat, #software, #technology, #tiktok, #united-states

This Week in Apps: WWDC 21 highlights, Instagram Creator Week recap, Android 12 beta 2 arrives

Welcome back to This Week in Apps, the weekly TechCrunch series that recaps the latest in mobile OS news, mobile applications and the overall app economy.

The app industry continues to grow, with a record 218 billion downloads and $143 billion in global consumer spend in 2020. Consumers last year also spent 3.5 trillion minutes using apps on Android devices alone. And in the U.S., app usage surged ahead of the time spent watching live TV. Currently, the average American watches 3.7 hours of live TV per day, but now spends four hours per day on their mobile devices.

Apps aren’t just a way to pass idle hours — they’re also a big business. In 2019, mobile-first companies had a combined $544 billion valuation, 6.5x higher than those without a mobile focus. In 2020, investors poured $73 billion in capital into mobile companies — a figure that’s up 27% year-over-year.

This week, our series will take a dive into the key announcements impacting app developers from WWDC 21.

This Week in Apps will soon be a newsletter! Sign up here: techcrunch.com/newsletters

WWDC 21 Wrap-Up

Image Credits: Apple

Apple’s WWDC went virtual again this year, but it didn’t slow down the pace of announcements. This week, Apple introduced a slate of new developer tools and frameworks, changes to iOS that will impact how consumers use their devices and new rules for publishing on its App Store, among other things. We don’t have the bandwidth to dig into every dev update — and truly, there are better places to learn about, say, the new concurrency capabilities of Swift 5.5 or what’s new with SwiftUI.

But after a few days of processing everything new, here’s what’s jumping out as the bigger takeaways and updates.

Xcode Cloud

Apple’s development IDE, Xcode 13, now includes Xcode Cloud, a built-in continuous integration and delivery service hosted on Apple’s cloud infrastructure. Apple says the service, birthed out of its 2018 Buddybuild acquisition, will help to speed up the pace of development by combining cloud-based tools for building apps along with tools to run automated tests in parallel, deliver apps to testers via TestFlight and view tester feedback through the web-based App Store Connect dashboard. Beyond the immediate improvements to the development process (which developers are incredibly excited about based on #WWDC21 tweets) Xcode Cloud represents a big step by Apple further into the cloud services space, where Amazon (AWS), Google and Microsoft have dominated. While Xcode Cloud may not replace solutions designed for larger teams with more diverse needs, it’s poised to make app development easier — and deliver a new revenue stream to Apple. If only Apple had announced the pricing! 

Swift Playgrounds 4

Image Credits: Apple

Swift Playgrounds got a notable update in iPadOS 15, as it will now allow developers to build iPhone and iPad apps right on their iPad and submit them to the App Store. In Swift Playgrounds 4, coming later this year, Apple says developers will be able to create the visual design of an app using SwiftUI, see the live preview of their app’s code while building and can run their apps full-screen to test them out. App projects can also be opened and edited with either Swift Playgrounds or Xcode.

While it’s not the Xcode on iPad system some developers have been requesting, it will make app building more accessible because of iPad’s lower price point compared with Mac. It could also encourage more people to try app development, as Swift Playgrounds helps student coders learn the basics then move up to more challenging lessons over time. Now, they can actually build real apps and hit the publish button, too.

App Store

Antitrust pressure swirling around Apple has contributed to a growing sentiment among some developers that Apple doesn’t do enough to help them grow their businesses — and therefore, is undeserving of a 15%-30% cut of the revenues the developers themselves worked to gain. The new App Store updates may start to chip away at that perception.

Soon, developers will be able to create up to 35 custom product pages targeted toward different users, each with their unique URL for sharing and analytics for measuring performance. The pages can include different preview videos, screenshots and text.

Image Credits: Apple

Apple will also allow developers to split traffic between three treatments of the app’s default page to measure which ones convert best, then choose the percentage of the App Store audience that will see one of the three treatments.

Meanwhile, the App Store will begin to show to customers in-app events taking place inside developers’ apps — like game competitions, fitness challenges, film premieres and more — effectively driving traffic to apps and re-engaging users. Combined, Apple is making the case that its App Store can drive discovery beyond just offering an app listing page.

Beyond the App Store product itself, Apple overhauled its App Store policies to address the growing problem of scam apps. The changes give Apple permission to crack down on scammers by removing offenders from its Developer Program. The new guidelines also allow developers to report spam directly to Apple, instead of, you know, relying on tweets and press.

Apple has historically downplayed the scam problem. It noted how the App Store stopped over $1.5 billion in fraudulent transactions in 2020, for example. Even if it’s a small percentage of the App Store, scam apps with fake ratings not only can cheat users out of millions of dollars, they reduce consumer trust in the App Store and Apple itself, which has longer-term consequences for the ecosystem health. What’s unclear, however, is why Apple is seemingly trying to solve the App Review issues using forms — to report fraud (and now, to appeal rulings, too) when it’s becoming apparent that Apple needs a more systematic way of keeping tabs on the app ecosystem beyond the initial review process.

Notifications overhaul

The App Store discovery updates mentioned above also matter more because developers may need to reduce their reliance on notifications to send users back into their apps. Indeed, iOS 15 users will be able to choose which apps they don’t need to hear from right away — these will be rounded up into a new Notification Summary that arrives on a schedule they configure, where Siri intelligence helps determine which apps get a top spot. If an app was already struggling to re-engage users through push notifications, getting relegated to the end of a summary is not going to help matters.

And users can “Send to Summary” right from the Lock Screen notification itself in addition to the existing options to “Deliver Quietly” or be turned off. That  means any ill-timed push could be an app developer’s last.

Image Credits: Apple

Meanwhile, the clever new “Focus” modes let iOS users configure different quiet modes for work, play, sleeping and more, each with their own set of rules and even their own home screens. But making this work across the app ecosystem will require developer adoption of four “interruption levels,” ranging from passive to critical. A new episode of a fav show should be a “passive” notification, for example. “Active” is the default setting — which doesn’t get to break into Focus. “Time sensitive” notifications should be reserved for alerting to more urgent matters, like a delivery that’s arrived on your doorstep or an account security update. These may be able to break through Focus, if allowed.

Image Credits: Apple

“Critical” notifications would be reserved for emergencies, like severe weather alerts or local safety updates. While there is a chance developers may abuse the new system to get their alert through, they risk users silencing their notifications entirely or deleting the app. Focus mode users will be power users and more technically savvy, so they’ll understand that an errant notification here was a choice and not a mistake on the developer’s part.

Image Credits: Apple

Augmented Reality

Apple has been steadily pushing out more tools for building augmented reality apps, but this WWDC it just introduced a huge update that will make it easier for developers getting started with AR. With the launch of RealityKit 2, Apple’s new Object Capture API will allow developers to create 3D models in minutes using only an iPhone or iPad (or a DSLR or drone if they choose).

Explains Apple this will address one of the most difficult parts of making great AR apps, which was the process of creating 3D models. Before, this could take hours and cost thousands of dollars — now, developers with just an iPhone and Mac can participate. The impacts of this update will be seen in the months and years ahead, as developers adopt the new tools for things like AR shopping, games and other AR experiences — including ones we may not have seen yet, but are enabled by more accessible AR technology tools and frameworks.

SharePlay

This update is unexpected and interesting, despite missing what would have been an ideal launch window: mid-pandemic back in 2020. With SharePlay, developers can bring their apps into what Apple is calling “Group Activities” — or shared experiences that take place right inside FaceTime.

If you were co-watching Hulu with friends during the pandemic, you get the idea. But Apple isn’t tacking on some co-viewing system here. Instead, it’s introducing new APIs that let users listen to music, stream video or screen share with friends, in a way that feels organic to FaceTime. There was a hint of serving the locked-down COVID-19 pandemic crowd with this update, as Apple talks about making people feel as if they’re “in the same room” — a nod to those many months where that was not possible. And that may have inspired the changes, to be sure. Similarly, FaceTime’s support for Android and scheduled calls — a clear case of Zoom envy — feels like a case of playing catch-up on Apple’s part.

Image Credits: Apple

The immediate demand for these sorts of experiences may be dulled by a population that’s starting to recover from the pandemic — people are now going out and seeing others in person again thanks to vaccines. But the ability to use apps while FaceTime’ing has a lifespan that extends beyond the COVID era, particularly among iPhone’s youngest users. The demographic growing up with smartphones at ever-younger ages don’t place phone calls — they text and FaceTime. Some argue Gen Z even prefers the latter.

Image Credits: Apple

With its immediate support for Apple services like Apple Music and Apple TV+, SharePlay will hit the ground running — but it will only fully realize its vision with developer adoption. But such a system seems possibly only because of Apple’s tight control over its platform. It also gives a default iOS app a big advantage over third-parties.

More

There were, of course, hundreds of updates announced this week, like Spatial audio, Focus modes, AirPods updates, iPadOS improvements (widgets! multi-tasking), Health updates, iCloud+ with Private Relay, watchOS improvements, Spotlight’s upgrade, macOS 12 Monterey (with Continuity with Universal Control), HomePod updates, StoreKit 2, Screen Time APIs, ShazamKit, App Clips improvements, Photos improvements and others.

Many, however, were iterative updates — like a better version Apple Maps, for example, or Siri support for third-party devices. Others are Apple’s attempt to catch up with competitors, like the Google Lens-like “Live Text” update for taking action on things snapped in your photos. The more significant changes, however, aren’t yet here — like the plan to add Driver’s Licenses to Wallet and the plan to shift to passwordless authentication systems. These will change how we use devices for years to come.

Weekly News

Platforms: Google

✨ Not to be outdone by WWDC (ha), Google this week launched Android 12, beta 2. This release brings more of the new features and design changes to users that weren’t yet available in the first beta which debuted at Google I/O. This includes the new privacy dashboard; the addition of the mic and camera indicators that show when an app is using those features; an indication when an app is reading from the clipboard; and a new panel that makes it easier to switch between internet providers or Wi-Fi networks.

Google also this week released its next Pixel feature drop which brought new camera and photo features, privacy features, Google Assistant improvements and more. Highlights included a way to create stargazing videos, a car crash detection feature and a way to answer or reject calls hands-free.

E-commerce

Pinterest wants to get more users clicking “buy.” The company this week added a new Shopping List feature which automatically organizes your saved Product Pins for easier access.

Augmented Reality

Google discontinued its AR-based app Measure, which had allowed users to measure things in the real world using the phone’s camera. The app had seen some stability and accuracy issues in the past.

Fintech

Facebook’s Messenger app added Venmo-like QR codes for person-to-person payments inside its app in the U.S. Users can scan the codes to send or request a payment, even if they’re not Facebook friends with the other party. Payments are sent over Facebook Pay, which is backed by a users’ credit card, debit card or a PayPal account.

Downloads of fintech apps are up 132% globally YoY according to an AppsFlyer marketing report.

Twitter and Square CEO Jack Dorsey said Square is thinking about adding a bitcoin hardware wallet to its product lineup. The exec detailed some of the thinking behind the plan in a Twitter thread.

✨ Social: Creator Week recap

Instagram head Adam Mosseri said Facebook will help creators get around Apple’s 30% cut. While any transactions that take place in iOS will follow Apple’s rules, Mosseri said Facebook will look for other ways to help creators make a living where they don’t have to give up a portion of their revenue — like by connecting brands and creators offline or affiliate deals.

Related to this, Instagram announced during its Creator Week event it will start testing a native affiliate tool that will allow creators to recommend products and then earn commissions on those sales. Creators can also now link their merch shops to personal profiles instead of just business profiles, and by year-end, will be able to partner on merch and drops with companies like Bravado/UMG, Fanjoy, Represent and Spring.

Image Credits: Instagram

Instagram also rolled out a new “badge” for live videos which lets viewers tip creators, similar to Facebook’s Stars. Facebook also said paid online events, fan subscriptions, badges and its upcoming news products will remain free through 2023. And it rolled out new features and challenges to help creators earn additional payouts for hitting certain milestones.

Finally, Instagram in a blog post explained how its algorithm works. The post details how the app decides what to show users first, why some posts get more views than others, how Explore works and other topics.

Messaging

Giphy’s Clips (GIFs with sound) are now available in the Giphy iMessage app, instead of only on the web and in its iOS app. That means you can send the…uh, videos (??)…right from your keyboard.

Dating

Image Credits: Tinder

Match-owned dating app Tinder added a way for users to block contacts. The feature requires users grant the app permission to access the phone’s contacts database, which is a bit privacy-invasive. But then users can go through their contacts and check those they want to block on Tinder. The benefit is this would allow people to block exes and abusers. But on the downside, it permits cheating as users can block partners and those who might see them and report back.

Streaming & Entertainment

YouTube will allow creators to repurpose audio from existing YouTube videos as its “Shorts” product — basically, its TikTok competitor — rolls out to more global markets.

Gaming

Google’s cross-platform cloud gaming service Google Stadia is coming to Chromecast with Google TV and Android TV starting on June 23.

Roblox is generating estimated revenue of $3.01 million daily on iPhone, according to data from Finbold. Clash of Clans, Candy Crush Saga, Pokémon GO and others follow. Good thing if they have to pay up over that music usage lawsuit.

Image Credits: Finbold

Utilities

Apple-owned weather app Dark Sky, whose technology just powered a big iOS 15 revamp of Apple’s stock weather app, is not shutting down just yet. The company announced its iOS app, web app and API will remain online through the end of 2022, instead of 2021 as planned.

Productivity

Microsoft’s Outlook email app for iOS now lets you use your voice to write emails and schedule meetings. The feature leverages Cortana, and follows the launch of a Play My Emails feature inside Outlook Mobile.

Government & Policy

President Biden revoked and replaced Trump’s actions which had targeted Chinese apps, like TikTok and WeChat. The president signed a new executive order that requires the Commerce Dept. to review apps with ties to “foreign adversaries” that may pose national security risks. Trump had previously tried to ban the apps outright, but his order was blocked by federal courts.

Google has agreed to show more mobile search apps for users to choose from on new Android phones following feedback from the European Commission. The company had been showing a choice screen where app providers bid against each other for the slot, and pay only if users download apps. DuckDuckGo and others complained the solution has not been working.

Security & Privacy

Security flaws were found in Samsung’s stock mobile apps impacting some Galaxy devices. One could have allowed for data theft through the Secure Folder app. Samsung Knox security software could have been used to install malicious apps. And a bug in Samsung Dex could have scraped data from notifications. There are no indications users were impacted and the flaws were fixed.

An App Store analysis published by The Washington Post claims nearly 2% of the top grossing apps on one day were scam apps, which cost people $48 million. They included several VPN apps that told users their iPhones were infected with viruses, a QR code reader that tricked customers into a subscription for functionality that comes with an iPhone, and apps that pretend to be from big-name brands, like Amazon and Samsung.

Multiple apps were removed from the Chinese app store for violating data collection rules, Reuters reported. The apps hailed from Sogou, iFlytek and others, and included virtual keyboards.

Funding and M&A

?Mexican payments app Clip raised $250 million from SoftBank’s Latin American Fund and Viking Global Investors, valuing the business at $2 billion. The app offers a Square-like credit card reader device and others, and has begun to offer cash advances to clients.

? Shopify acqui-hires the team from the augmented reality home design app Primer. The app, which will be shut down, had allowed users to visualize what tile, wallpaper or paint will look like on surfaces inside their home.

? Singapore-based corporate services “super app” Osome raised $16 million in Series A funding. The app offers online accounting and other business services for SMBs. Investors include Target Global, AltaIR Capital, Phystech Ventures, S16VC and VC Peng T. Ong.

?  Chinese grocery delivery app Dingdong Maicai, backed by Sequoia and Tiger Global, has filed for a U.S. IPO. To date, the company has raised $1 billion.

?San Francisco-based MaintainX raised $39 million in Series B funding led by Bessemer Venture Partners for its mobile-first platform for industrial and frontline workers to help track maintenance, safety and operations.

?Berlin’s Ada Health raised $90 million in Series B funding in a round led by Leaps by Bayer, the impact investment arm of Bayer AG. The app lets users monitor their symptoms and track their health and clinical data.

?Photo app Dispo confirmed its previously leaked Series A funding, which earlier reports had pegged as being around $20 million. The app had been rebranded from David’s Disposable and dropped its association with YouTuber David Dobrik, following sexual assault allegations regarding a member of the Vlog Squad. Spark Capital severed ties with Dispo as a result. Seven Seven Six and Unshackled Ventures remained listed as investors, per Dispo’s press release, but the company didn’t confirm the size of the round.

?Brazilian fintech Nubank raised a $750 million extension to its Series G (which was $400 million last year) led by Berkshire Hathaway. The company offers a digital bank account accessible from an app, debit card, payments, loans, insurance and more. The funding brings the company to a $1.15 billion valuation.

?Seattle-based tutoring app Kadama raised $1.7 million in seed funding led by Grishin Robotics. The app, which offering an online tutoring marketplace aimed at Gen Z, rode the remote learning wave to No. 2 in the Education category on the App Store.

?  Mark Cuban-based banking app Dave, which helps Americans build financial stability, is planning to go public via a SPAC launched by Chicago-based Victory Park Capital called VPC Impact Acquisition Holdings III. It also includes a $210 million private investment from Tiger Global Management.

?  Mobile game publisher Voodoo acquired Tel Aviv-based marketing automation platform Bidshake for an undisclosed sum. Launched in January 2020, Bidshake combines data aggregation and analytics with campaign and creative management. It will continue to operate independently.

Downloads

Turntable — tt.fm

Image Credits: tt.fm on iPhone/Brian Heater

Newly launched music social network tt.fm is a Turntable.fm rival that lets you virtually hang out with friends while listening to music. To be clear, the app is not the same as Turntable.fm, which shut down in 2013 but then returned during the pandemic as people looked to connect online. While that Turntable was rebirthed by its founder Billy Chasen, Turntable – tt.fm hails from early Turntable.fm employee, now tt.fm CEO Joseph Perla. But as live events are coming back, the question now may be not which Turntable app to choose, but whether the Turnable.fm experience has missed the correct launch window…again.

SketchAR

SketchAR

SketchAR

The art app SketchAR previously offered artists tools to draw with AR, turn photos into AR, create AR masks for Snapchat, play games and more. With its latest update, artists can now turn their work into NFTs directly inside the app and sell it. The app, now used by nearly 500,000 users, will select a “Creator of the Week” to NFT on OpenSea. Others can create and auction their art as NFTs on-demand.

Tweets

#android, #android-apps, #app-stores, #apple, #apps, #ar, #augmented-reality, #creators, #developers, #facebook, #google, #instagram, #ios, #ios-apps, #ipad, #ipados, #mobile, #swift, #tc, #this-week-in-apps, #wwdc, #wwdc-2021, #xcode

Apple’s RealityKit 2 allows developers to create 3D models for AR using iPhone photos

At its Worldwide Developer Conference, Apple announced a significant update to RealityKit, its suite of technologies that allow developers to get started building AR (augmented reality) experiences. With the launch of RealityKit 2, Apple says developers will have more visual, audio, and animation control when working on their AR experiences. But the most notable part of the update is how Apple’s new Object Capture technology will allow developers to create 3D models in minutes using only an iPhone.

Apple noted during its developer address that one of the most difficult parts of making great AR apps was the process of creating 3D models. These could take hours and thousands of dollars.

With Apple’s new tools, developers will be able take a series of pictures using just an iPhone (or iPad or DSLR, if they prefer) to capture 2D images of an object from all angles, including the bottom.

Then, using the Object Capture API on macOS Monterey, it only takes a few lines of code to generate the 3D model, Apple explained.

Image Credits: Apple

To begin, developers would start a new photogrammetry session in RealityKit that points to the folder where they’ve captured the images. Then, they would call the process function to generate the 3D model at the desired level of detail. Object Capture allows developers to generate the USDZ files optimized for AR Quick Look — the system that lets developers add virtual, 3D objects in apps or websites on iPhone and iPad. The 3D models can also be added to AR scenes in Reality Composer in Xcode.

Apple said developers like Wayfair, Etsy and others are using Object Capture to create 3D models of real-world objects — an indication that online shopping is about to get a big AR upgrade.

Wayfair, for example, is using Object Capture to develop tools for their manufacturers so they can create a virtual representation of their merchandise. This will allow Wayfair customers to be able to preview more products in AR than they could today.

Image Credits: Apple (screenshot of Wayfair tool))

In addition, Apple noted developers including Maxon and Unity are using Object Capture for creating 3D content within 3D content creation apps, such as Cinema 4D and Unity MARS.

Other updates in RealityKit 2 include custom shaders that give developers more control over the rendering pipeline to fine tune the look and feel of AR objects; dynamic loading for assets; the ability to build your own Entity Component System to organize the assets in your AR scene; and the ability to create player-controlled characters so users can jump, scale and explore AR worlds in RealityKit-based games.

One developer, Mikko Haapoja of Shopify, has been trying out the new technology (see below) and shared some real-world tests where he shot objects using an iPhone 12 Max via Twitter.

Developers who want to test it for themselves can leverage Apple’s sample app and install Monterey on their Mac to try it out.

read more about Apple's WWDC 2021 on TechCrunch

#animation, #apple, #apple-inc, #apps, #ar, #augmented-reality, #computing, #ios, #ipad, #iphone, #macos, #mobile, #online-shopping, #realitykit, #unity, #wayfair, #wwdc-2021

Facebook’s Spark AR platform expands to video calling with Multipeer API

At today’s F8 developer conference, Facebook announced new capabilities for Spark AR, its flagship AR creation software. Since Spark AR was announced at F8 2017, more than 600,000 creators from 190 countries have published over 2 million AR effects on Facebook and Instagram, making it the largest mobile AR platform, according to Facebook. If you’ve ever posted a selfie on your Instagram story with an effect that gave you green hair, or let you control a dog’s facial expression by moving your own face, then you’ve used Spark AR

Soon, these AR effects will be available for video calling on Messenger, Instagram, and Portal with the introduction of a Multipeer API. Creators can develop effects that bring call participants together by using a shared AR effect. As an example, Spark AR shared a promo video of a birthday party held over a video call, in which an AR party hat appears on each of the participants’ heads. 

Creators can also develop games for users to play during their video calls. This already exists on Facebook video calls – think of the game where you compete to see who can catch the most flying AR hamburgers in their mouth in a minute. But when the ability to make new, lightweight games opens to developers, we’ll see some new games to challenge our friends with on video calls. 

These video call effects and multipeer AR games will be bolstered by Spark’s platform exclusive multi-class segmentation capability. This lets developers augment multiple segments of a user’s body (like hair or skin) at once within a single effect. 

Facebook also discussed its ongoing ambition to build AR glasses. Chris Barber, Director of Partnerships for Spark AR, said that this goal is still “years away” – but, Barber did tease some potential features for the innovative, wearable tech. 

“Imagine being able to teleport to a friend’s sofa to watch a show together, or being able to share a photo of something awesome you see on a hike,” Barber said. Maybe this won’t sound so dystopian by the time the product launches, years down the road. 

Last October, Spark AR launched the AR Partner Network, a program for the platform’s most advanced creators, and this year, Spark launched an AR curriculum through Facebook’s BluePrint Platform to help creators learn how to improve their AR effects. Applications for the Spark Partner Network will open again this summer. For now, creators and developers can apply to start building effects for video calling through the Spark AR Video Calling Beta

#api, #apps, #ar, #augmented-reality, #facebook, #instagram, #messenger, #mobile-software, #operating-systems, #social, #social-media, #software, #spark, #spark-ar

Snap acquires AR startup WaveOptics, which provides tech for Spectacles, for over $500M

Snap yesterday announced the latest iteration of its Spectacles augmented reality glasses, and today the company revealed a bit more news: it is also acquiring the startup that supplied the technology that helps power them. The Snapchat parent is snapping up WaveOptics, an AR startup that makes the waveguides and projectors used in AR glasses. These overlay virtual images on top of the views of the real world someone wearing the glasses can see, and Snap worked with WaveOptics to build its latest version of Spectacles.

The deal was first reported by The Verge, and a spokesperson for Snap directly confirmed the details to TechCrunch. Snap is paying over $500 million for the startup, in a cash-and-stock deal. The first half of that will be coming in the form of stock when the deal officially closes, and the remainder will be payable in cash or stock in two years.

This is a big leap for WaveOptics, which had raised around $65 million in funding from investors that included Bosch, Octopus Ventures and a host of individuals, from Stan Boland (veteran entrepreneur in the UK, most recently at FiveAI) and Ambarish Mitra (the co-founder of early AR startup Blippar). PitchBook estimates that its most recent valuation was only around $105 million.

WaveOptics was founded in Oxford, and it’s not clear where the team will be based after the deal is closed — we have asked.

We have been covering the company since its earliest days, when it displayed some very interesting, early, and ahead-of-its-time technology: waveguides based on hologram physics and photonic crystals. The important and key thing is that its tech drastically compresses size and load of the hardware needed to process and display images, meaning a much wider and more flexible range of form factors for AR hardware based on WaveOptics tech.

It’s not clear whether WaveOptics will continue to work with other parties post-deal, but it seems that one obvious advantage for Snap would be making the startup’s technology exclusive to itself.

Snap has been on something of an acquisition march in recent times — it’s made at least three other purchases of startups since January, including Fit Analytics for an AR-fuelled move into e-commerce, as well as Pixel8Earth and StreetCred for its mapping tools.

This deal, however, marks Snap’s biggest acquisition to date in terms of valuation. That is not only a mark of the premium price that foundational artificial intelligence tech continues to command — in addition to the team of scientists that built WaveOptics, it also has 12 filed and in-progress patents — but also Snap’s financial and, frankly, existential commitment to having a seat at the table when it comes not just to social apps that use AR, but hardware, and being at the centre of not just using the tech, but setting the pace and agenda for how and where that will play out.

That’s been a tenacious and not always rewarding place for it to be, but the company — which has long described itself as a “camera company” — has kept hardware in the mix as an essential component for its future strategy.

 

#ar, #artificial-intelligence, #augmented-reality, #computer-vision, #europe, #exit, #glasses, #snap, #snapchat, #spectacles, #tc, #waveguides, #waveoptics

Snap debuts true AR glasses that show the potential (and limitations) of AR

Snap Inc., the company best known for the popular Snapchat social camera app, has announced its first pair of augmented reality glasses that most people would agree actually qualify as real AR glasses. Like previous glasses the company has produced, they are called Spectacles.

Spectacles will not be available to buy as a mass-market consumer product—at least not in the immediately foreseeable future. Instead, Snap is seeding units to developers and content creators so the glasses can be used to create new experiences and filters. These creators will build these with Lens Studio, a Snapchat-specific tool that is already widely in use.

Spectacles enable new ways to view and create Snapchat Lenses, which are generally simple augmented reality filters that Snapchat users apply to the videos they send each other.

Read 16 remaining paragraphs | Comments

#ar, #ar-glasses, #augmented-reality, #filters, #glasses, #snap, #snap-spectacles, #snapchat, #snapchat-spectacles, #spectacles, #tech

Google Maps to add more detailed maps, crowd indicators, better routing and more

Google has announced a series of updates soon coming to Google Maps, as part of the company’s larger goal of delivering over 100 A.I.-powered improvements to the platform by year-end. Among the new improvements, detailed during Google I/O’s developer conference this week, are new routing updates, Live View enhancements, an expansion of detailed street maps, a new “area busyness” feature, and a more personalized Maps experience.

The new routing updates will involve the use of machine learning and navigation information to help reduce “hard-braking moments” — meaning, those times when traffic suddenly slows, and you have to slam on your brakes.

Today, when you get your directions in Maps, Google calculates multiple route options based on a variety of factors, like how many lanes a road has or how direct the route is. With the update, it will add one more: which routes are least likely to cause a “hard-braking moment.” Google will recommend the route that has the least likelihood of those sorts of moments, if the ETA is the same or the difference is minimal between another route. The company says it expects this change could potentially eliminate 100 million hard-breaking events in routes driven with Google Maps every year.

Live View, Google Maps’ augmented reality feature launched in 2019, will soon become available directly from the map interface so you can instantly explore the neighborhood and view details about nearby shops and restaurants, including how busy they are, recent reviews and photos. It will also be updated to include street signs for complex intersections, and will tell you where you are in relation to places like your hotel, so you can make your way back more easily, when in unfamiliar territory.

Image Credits: Google

Google will also expand the more detailed maps it first rolled out to last year to New York, San Francisco, and London. These maps offer more granularity, including both natural features and street info like the location of sidewalks, crosswalks and pedestrian islands, for example. The information can be particularly useful for those who navigate a city by foot, scooter, bike, or in a wheelchair.

By the end of 2021, these detailed maps will be available in 50 more cities, including  Berlin, São Paulo, Seattle, and Singapore.

Image Credits: Google

Another new feature expands on the “busyness” information Google already provided for businesses, based on anonymized location data collected by Maps users. During the pandemic, that feature became a useful way to avoid crowds at local stores and other businesses, for health and safety. Now, Google Maps will display “busyness” info for parts of town or neighborhoods, to help you either avoid (or perhaps locate) crowded areas — like a street festival, farmers’ market, or nightlife spot, among other things.

Image Credits: Google

Finally, Google Maps will begin customizing its interface to the individual in new ways.

For starters, it may show relevant information based on the time of day where you are.

For instance, when you open the map at 8 AM on a weekday, you may see coffee shops more prominently highlighted, but at night, you may see dinner spots. If you’ve traveled out of town, Google Maps may instead show you landmarks and tourists attractions. And if you want to see more of the same, you can tap on any place to see similar places nearby.

Image Credits: Google

 

Google says these features will roll out globally across iOS and Android in the coming months, but did not provide an exact timeframe for each specific feature. The more detailed maps will arrive by year-end, however.

#apps, #ar, #artificial-intelligence, #augmented-reality, #cities, #crowds, #google, #google-maps, #machine-learning, #maps, #navigation, #streets, #tc

SightCall raises $42M for its AR-based visual assistance platform

Long before Covid-19 precipitated “digital transformation” across the world of work, customer services and support was built to run online and virtually. Yet it too is undergoing an evolution supercharged by technology.

Today, a startup called SightCall, which has built an augmented reality platform to help field service teams, the companies they work for, and their customers carry out technical and mechanical maintenance or repairs more effectively, is announcing $42 million in funding, money that it plans to use to invest in its tech stack with more artificial intelligence tools and expanding its client base.

The core of its service, explained CEO and co-founder Thomas Cottereau, is AR technology (which comes embedded in their apps or the service apps its customers use, with integrations into other standard software used in customer service environments including Microsoft, SAP, Salesforce and ServiceNow). The augmented reality experience overlays additional information, pointers and other tools over the video stream.

This is used by, say, field service engineers coordinating with central offices when servicing equipment; or by manufacturers to provide better assistance to customers in emergencies or situations where something is not working but might be repaired quicker by the customers themselves rather than engineers that have to be called out; or indeed by call centers, aided by AI, to diagnose whatever the problem might be. It’s a big leap ahead for scenarios that previously relied on work orders, hastily drawn diagrams, instruction manuals, and voice-based descriptions to progress the work in question.

“We like to say that we break the barriers that exist between a field service organization and its customer,” Cottereau said.

The tech, meanwhile, is unique to SightCall, built over years and designed to be used by way of a basic smartphone, and over even a basic mobile network — essential in cases where reception is bad or the locations are remote. (More on how it works below.)

Originally founded in Paris, France before relocating to San Francisco, SightCall has already built up a sizable business across a pretty wide range of verticals, including insurance, telecoms, transportation, telehealth, manufacturing, utilities, and life sciences/medical devices.

SightCall has some 200 big-name enterprise customers on its books, including the likes of Kraft-Heinz, Allianz, GE Healthcare and Lincoln Motor Company, providing services on a B2B basis as well as for teams that are out in the field working for consumer customers, too. After seeing 100% year-over-year growth in annual recurring revenue in 2019 and 2020, SightCall’s CEO says it’s looking like it will hit that rate this year as well, with a goal of $100 million in annual recurring revenue.

The funding is being led by InfraVia, a European private equity firm, with Bpifrance also participating. The valuation of this round is not being disclosed, but I should point out that an investor told me that PitchBook’s estimate of $122 million post-money is not accurate (we’re still digging on this and will update as and when we learn more).

For some further context on this investment, InfraVia invests in a number of industrial businesses, alongside investments in tech companies building services related to them such as recent investments in Jobandtalent, so this is in part a strategic investment. SightCall has raised $67 million to date.

There has been an interesting wave of startups emerging in recent years building out the tech stack used by people working in the front lines and in the field, a shift after years of knowledge workers getting most of the attention from startups building a new generation of apps.

Workiz and Jobber are building platforms for small business tradespeople to book jobs and manage them once they’re on the books; BigChange helps manage bigger fleets; and Hover has built a platform for builders to be able to assess and estimate costs for work by using AI to analyze images captured by their or their would-be customers’ smartphone cameras.

And there is Streem, which I discovered is a close enough competitor to SightCall that they’ve acquired Adwords ads based on SightCall searches in Google. Just ahead of the Covid-19 pandemic breaking wide open, General Catalyst-backed Streem was acquired by Frontdoor to help with the latter’s efforts to build out its home services business, another sign of how all of this is leaping ahead.

What’s interesting in part about SightCall and sets it apart is its technology. Co-founded in 2007 by Cottereau and Antoine Vervoort (currently SVP of product and engineering), the two are both long-time telecoms industry vets who had both worked on the technical side of building next-generation networks.

SightCall first started life as a company called Weemo that built video chat services that could run on WebRTC-based frameworks, which emerged at a time when we were seeing a wider effort to bring more rich media services into mobile web and SMS apps. For consumers and to a large extent businesses, mobile phone apps that work ‘over the top’ (distributed not by your mobile network carrier but the companies that run your phone’s operating system, and thus partly controlled by them) really took the lead and continue to dominate the market for messaging and innovations in messaging.

After a time, Weemo pivoted and renamed itself as SightCall, focusing on packaging the tech that it built into whichever app (native or mobile web) where one of its enterprise customers wanted the tech to live.

The key to how it works comes by way of how SightCall was built, Cottereau explained. The company has spent ten years building and optimizing a network across data centers close to where its customers are, which interconnects with Tier 1 telecoms carriers and has a lot of latency in the system to ensure uptime. “We work with companies where this connectivity is mission critical,” he said. “The video solution has to work.”

As he describes it, the hybrid system SightCall has built incorporates its own IP that works both with telecoms hardware and software, resulting in a video service that provides 10 different ways for streaming video and a system that automatically chooses the best in a particular environment, based on where you are, so that even if mobile data or broadband reception don’t work, video streaming will. “Telecoms and software are still very separate worlds,” Cottereau said. “They still don’t speak the same language, and so that is part of our secret sauce, a global roaming mechanism.”

The tech that the startup has built to date not only has given it a firm grounding against others who might be looking to build in this space, but has led to strong traction with customers. The next steps will be to continue building out that technology to tap deeper into the automation that is being adopted across the industries that already use SightCall’s technology.

“SightCall pioneered the market for AR-powered visual assistance, and they’re in the best position to drive the digital transformation of remote service,” said Alban Wyniecki, partner at InfraVia Capital Partners, in a statement. “As a global leader, they can now expand their capabilities, making their interactions more intelligent and also bringing more automation to help humans work at their best.”

“SightCall’s $42M Series B marks the largest funding round yet in this sector, and SightCall emerges as the undisputed leader in capital, R&D resources and partnerships with leading technology companies enabling its solutions to be embedded into complex enterprise IT,” added Antoine Izsak of Bpifrance. “Businesses are looking for solutions like SightCall to enable customer-centricity at a greater scale while augmenting technicians with knowledge and expertise that unlocks efficiencies and drives continuous performance and profit.”

Cottereau said that the company has had a number of acquisition offers over the years — not a surprise when you consider the foundational technology it has built for how to architect video networks across different carriers and data centers that work even in the most unreliable of network environments.

“We want to stay independent, though,” he said. “I see a huge market here, and I want us to continue the story and lead it. Plus, I can see a way where we can stay independent and continue to work with everyone.”

#ai, #ar, #artificial-intelligence, #augmented-reality, #customer-service, #enterprise, #europe, #field-service, #funding, #industrial, #manufacturing, #service-engineers, #sightcall, #tc, #weemo

Apple takes on Tile with AR-ready AirTags tracking devices

Carolyn Wolfman-Estrada, engineering program manager at Apple, presents AirTags (with one visible in her right hand).

Enlarge / Carolyn Wolfman-Estrada, engineering program manager at Apple, presents AirTags (with one visible in her right hand). (credit: Apple)

In a now-rare announcement of a completely new product category, Apple today introduced AirTags, a Tile-like personal location device.

AirTags can be placed in or on personal possessions to be tracked with the Find My app (formerly Find My iPhone) on iPhones, iPads, or Macs. Users can then find those devices, including those detected by any other Apple devices nearby.

The new devices play off Apple’s work in bringing augmented reality features to its devices. Users will be able to lift their phone cameras and see the locations of their AirTags positioned accurately in real physical space on the screen. Like some other similar products, AirTags will also be able to emit a noise to make them easier to find.

Read 3 remaining paragraphs | Comments

#airtags, #apple, #ar, #bluetooth, #gps, #tech, #u1, #ultra-wideband, #uwb