Large language models capable of providing statistically likely answers to written text prompts are transforming knowledge work. AI algorithms enable tasks to be performed faster and reduce the need for specialist skills, which can be unnerving to witness for highly trained and experienced staff. But there’s magic to how neural networks can level the playing field and help users to understand their world. And a great example of this is the rise of vision language models (VLMs), which co-locate image features alongside multi-dimensional word embeddings.

So-called multimodal AI – neural networks capable of generating results based on multiple input types such as text and images – adds to the number of ways that deep learning can be used. OpenAI’s GPT-4 with Vision will answer questions about images – for example, if the user submits a photo or web link. Given a picture of the contents of a fridge, it can suggest ideas for dinner.

READ NEXT

Like the sound of private, air-gapped LLMs? Welcome to generative AI on CPUs.

You can picture VLMs as image recognition on steroids. Multimodel AI models can do much more than just recognize features in a scene. Algorithms can gather insights from security footage, prompt maintenance crews to take action, analyse customer behaviour, and much more besides (as our test results show – scroll down to see the responses).

VLMs can provide a second pair of eyes that never get tired. However, these superpowers come at a cost and historically have required heavy lifting to happen behind the scenes in the cloud. Generative AI algorithms have gorged themselves on the contents of the internet in the name of fine-tuning their model weights.

But not all of those parameters are created equal, and developers are finding ways to trim AI models down to a more memory-efficient size for specific tasks. It means that algorithms can run locally at the edge and suit fully remote use cases such as drone inspections – for example, where connectivity cannot be guaranteed.

Releasing moondream2 – a small, open-source, vision language model designed to run efficiently on edge devices. Clocking in at 1.8B parameters, moondream requires less than 5GB of memory to run in 16 bit precision. pic.twitter.com/ySfmK4mJbN

— vik (@vikhyatk) March 4, 2024

One of a new breed of tiny VLMs capable of running at the edge is Moondream, which is advertised as being able to run anywhere, even on mobile devices and Raspberry Pi’s. The code to run the tiny AI text and vision model is available on GitHub, and there’s even a live demo environment so that you can jump straight in and put the VLM to the test.

Tiny VLM test results

To discover what VLMs can do and highlight potential use cases for local AI models capable of answering questions about images, TechHQ submitted a variety of photos to the Moondream2 live demo hosted on Huggin Face.

Image 1

Harbour scene. Photo: JT.

Input: Is it safe to walk straight ahead?

Response: No, it is not safe to walk straight ahead. There is a dock with a red life ring attached to it, and a few boats are visible in the water. You need to be cautious and look out for any obstacles or hazards such as the dock, boats, or other watercraft, before walking forward.

Image 2

Stairway. Photo: JT.

Input: Is there something for me to hold on to?

Response: Yes, there is a railing on the stairs, and you can hold on to it while climbing to the top. The railing is attached to the wall, providing support for those ascending or descending the steps.

Input: Can you see an open window?

Response: Yes, there is an open window on the building in the background, and the sun is shining brightly illuminating the scene.

Image 3

Pedestrian crossing. Photo: JT.

Input: Is it safe for me to use the pedestrian crossing?

Response: No, it is not safe for you to use the pedestrian crossing, as the image shows a black car approaching the crossing. The car is likely not paying attention to the red light, which indicates that it’s not safe for pedestrians to cross at that moment.

Image 4

Road signs. Photo: JT.

Input: Should I go left or right to get to the airport?

Response: Right.

Input: Which way to the marina?

Response: Left.

Image 5

Bus stop. Photo: JT.

Input: Is there anywhere to catch a bus?

Response: Yes, there is a bus stop in the image, with benches for passengers to sit on and a covered area. A traffic cone is also present, and a bridge can be seen in the background.

Input: Where can I shelter if it rains?

Response: You can shelter at the bus stop if it rains, as there is a bus shelter nearby with benches and a roof providing protection from the elements.

Image 6

Luxury yacht. Photo: JT.

Input: What can you see?

Response: A helicopter is positioned on a ship, with a metal fence in the foreground. The background features a variety of elements, including buildings, trees, a road, a water body, and the sky.

Image 7

Coffee shop. Photo: JT.

Input: Is there anywhere to get a warm drink around here?

Response: Yes, there is a coffee shop nearby. The image shows a red building with the words “New Cut Coffee” on it, and a black fence. A black car is parked in the lot, and a white building is visible in the background. The sky is blue overhead.

Input: Does the coffee shop sell sandwiches?

Response: Yes.

Input: What else is there to eat at the coffee shop?

Response: In addition to coffee, the New Cut Coffee shop offers sandwiches and pastries. The image also features a building and a vehicle in the background.

Image 8

Web cam mode: the tiny VLM live demo can accept images via a connected webcam if browser permissions are given. Screengrab: JT.

Input: Can you see any ethernet ports?

Response: Yes, there are two visible ports on the device that the man is holding.

Verdict on tiny VLM testing

It’s hard not to be impressed with the performance of Moondream2 – a VLM that’s initialized with weights from SigLIP and Phi 1.5, and then trained using synthetic data generated by Mixtral.

READ NEXT

Edge AI: how to make deep learning more efficient

Evaluating the responses generated from TechHQ’s test sequence of images and text inputs, it’s clear that Moondream2 gets more right than it gets wrong, and is capable of noticing fine details that weary human eyes may miss. Also, as mentioned, the AI model’s small footprint is another winning feature – paving the way for inference to take place on edge devices that could include drones or robots.

VLMs could turn out to be incredibly useful in helping robots orientate themselves in the world and being able to run algorithms locally eliminates the need for network access, or at least provides redundancy in the case of outages.

A smart idea for smart glasses

Alongside its launch of the Quest 3 VR headset, Meta also announced a collaboration with Ray-Ban to produce an updated line of smart glasses. The product, which features Qualcomm’s weight-distributed Snapdragon AR platform, has some local voice control capabilities – for example, to instruct the spectacles to take images using the built-in ultra-wide 12MP camera. Users simply have to say, “Hey Meta, take a picture.”

Being able to couple those photos with a VLM running locally would turn the Ray-Ban smart glasses into a much more compelling product, and could bring scenes to life for visually impaired wearers without the need for pairing with a smartphone.

Vision assistance powered by edge-compatible VLMs could dramatically enhance the capabilities of digital camera-equipped devices. As the webcam image in our test sequence highlights, there’s the potential for algorithms to help with maintenance and repair tasks, distributing knowledge and expertise across the globe.

AI is doing for knowledge work what robotics has done for manufacturing, and it’s just the beginning.

The post Tiny VLMs bring AI text plus image vision to the edge appeared first on TechHQ.

There are a lot of factors to consider when it comes to choosing a smartphone, and bringing everything together is the chipset inside the handset. And so whenever a new mobile processor hits the market, which happened this week with the release by MediaTek of its Dimensity 9300 smartphone chip, it’s worth considering what this means for consumers.

Contrary to what marketing materials may suggest, there’s no such thing as the perfect processor for mobile devices. Instead, it’s about finding a sweet spot. Chip designers must maximize performance while paying attention to cost, thermal management, power consumption, and many other parameters.

READ NEXT

Secret formula behind Huawei Mate 60 Pro sales

What’s interesting is that different designers come to different conclusions on what that sweet spot looks like. And it means that before hitting the ‘buy now’ button, consumers may benefit from learning more about the chips inside their laptops, tablets, and smartphones.

The MediaTek 5G Dimensity 9300 is the latest in a flurry of mobile chip launches. New releases include Qualcomm’s Snapdragon 8 Gen 3 – unveiled at the firm’s developer conference in Maui – and Apple’s A17 Pro, its first to feature TSMC’s 3nm process, launched at the iPhone 15 event.

List of flagship mobile chips:

• MediaTek Dimensity 9300
• Qualcomm Snapdragon 8 Gen 3
• Apple A17 Pro
• Samsung Exynos 2400
• Google Tensor G3

Without a doubt, MediaTek has Qualcomm’s popular Snapdragon 8 Gen 3 in its sights. The chip maker dubs its 4 ultra-large cores and 4 big cores design as being ‘All-Big-Core’ compared with Qualcomm’s combination of 1 ultra-large core, 5 big cores, and 2 small cores.

In a pre-launch event attended by TechHQ and other members of the tech media, MediaTek’s Finbar Moynihan – who manages the firm’s global PR, marketing, and communications activities – explained the rationale behind the ‘All-Big-Core’ approach.

With smartphone applications becoming richer and placing more demands on processors, which includes bigger peaks at the high end, MediaTek’s designers wanted to create more headroom for on-chip compute. Moynihan describes the MediaTek Dimensity 9300 chip as being able to turn on big core performance when needed and power down quickly once processing demands have been met to save on energy.

Vivo X100 Series will be the first smartphone to launch with MediaTek's Dimensity 9300 SoC#vivo #vivox100 #vivox100pro #vivox100proplus pic.twitter.com/VyVuWfyTIO

— Gizinfo.com (@Gizinfo_) November 6, 2023

According to the firm, the combination of four performance cores (1x Ultra-Large Cortex-X4 @ 3.25GHz and 3x Cortex-X4 @ 2.85GHz) and four efficiency cores (4x Big Cortex-A720 @2.0GHz) provides 40% higher peak performance. So, if you’re a smartphone user who pushes your device to the limit, then – based on these numbers – it could be worth seeking out a handset featuring the MediaTek Dimensity 9300 chip.

Moynihan comments that the flagship segment is one of the bright spots in mobile currently. And it appears that Vivo’s X100 will be the first flagship smartphone series to feature MediaTek’s Dimensity 9300 mobile chip. Also, it’s possible that the device could move into tablets too, as OEM explore what having four big cores brings to the table.

Another point to consider when shopping for smartphones is AI. If you’re a big user of generative AI, then there’s a lot to like about the latest generation of mobile processors, which includes the Dimensity 9300 chip.

READ NEXT

Is running AI on CPUs making a comeback?

MediaTek reports that its device has comprehensive large language model (LLM) support for Meta’s Llama 2 as well as algorithms from Baidu and AI startup Baichuan Intelligent Technology, which highlights the significance of the Chinese market to the mobile chip developer.

Advanced hardware compression technology, which includes removing zero-weights and other techniques for lowering the memory requirements of running LLMs, make it possible to run inference on-chip – even for large models.

According to MediaTek, the Dimensity 9300 mobile chip – which features a dedicated AI processor – can perform image generation using Stable Diffusion (the popular text-to-image model) in less than 1 second. “It’s the most capable edge implementation that we have today,” said Moynihan in the briefing event.

Mobile edge computing

There are some great examples of how chip developers are bringing more computing power back to the device. For example, Apple’s neural engine is proving to be a game-changer when it comes to volumetric photogrammetry, which turns hundreds of high-resolution images into 3D digital assets for games and other simulation environments.

Having real-time photogrammetry running locally on iPhone Pro models is huge for 3D scanning companies such as KIRI Engine. The app developer has incorporated Apple’s object capture API into the latest version of its popular smartphone 3D scanning software. And it highlights the benefits that smartphone chips with mobile edge computing capabilities bring to users.

On-screen cues make the photogrammetry experience straightforward. “All you need to do is fill in the circle while walking around the object,” says Jack Wang, one of KIRI Engine’s three co-founders, in an online demo. “What’s really changing the game here is that everything runs locally on your iPhone – no cloud, no server, whatsoever.”

According to Wang, this was unimaginable even just a year ago. However, structuring the API around the iPhone’s neural engine and supporting the complex image processing with LiDAR data has opened the door to real-time photogrammetry on a smartphone.

READ NEXT

Privacy by design: smartphone vendors make security a selling point

He suspects that LiDAR helps with photo alignment. Post-camera processing is one of the most computing-intensive steps in rendering a 3D digital model from an object photographed from multiple angles.

Once smartphone users experience the benefits of mobile edge computing there’s no going back. In this example, it was Apple with its A17 Pro chip, but as we’ve seen with the release of the Dimensity 9300, MediaTek and other developers are also expanding on-device capabilities through custom silicon designs.

Competition to produce the best flagship mobile chip is strong. However, as mentioned above, each device maker will bring something different, which brings rewards for consumers who like to do the research before upgrading their smartphone.

The post MediaTek goes all-big on cores for Dimensity 9300 smartphone chip appeared first on TechHQ.

• Private AI was born out of VMware’s need, which the company realized would resonate well with other enterprises.
• With a private AI architecture, businesses can run their preferred AI models, proprietary or open-source, near their data.
• Private AI is expected to have huge take-up across the industry.

The 2023 VMware Explore in Singapore’s  concluded last week, and as anticipated, the conversations were mainly centered on AI, multi-cloud, and the edge. But while generative AI was heavily spoken about, VMware was more thrilled to discuss private AI – an architectural approach to AI within an organization’s practical privacy and compliance needs. 

READ NEXT

Like the sound of private, air-gapped LLMs? Welcome to generative AI on CPUs.

Simply put, VMware has found a way to bring compute capacity and AI models to where enterprise data is created, processed, and consumed, whether in a public cloud, enterprise data center, or at the edge. “When we started looking into generative AI, our lawyers had to stop us from using ChatGPT,” CEO Raghu Raghuram said in his keynote address. 

“These are hard problems to solve. Because if you think about what goes into designing an AI model, each stage has significant privacy implications. So we took our team of AI researchers and created VMware AI labs and put them to work on solving this problem,” Raghuram said, adding that most enterprises endure the same issues. 

VMware CEO, Raghu Raghuram during his keynote at VMware Explore Singapore 2023.

“Many of the CEOs I speak with are actively asking their legal teams to dig in and collaborate with IT to define a new set of privacy standards built for the complex nuances of generative AI. It’s a complicated undertaking, to say the least. Unless you solve these privacy problems, you won’t be able to enjoy the advantages of generative AI,” he said. 

During that time, VMware AI Labs collaborated closely with the company’s General Counsel, Amy Fliegelman Olli, and her legal team. Together, their engineers and lawyers sorted through the intricacies of choosing an AI model, training it using domain-specific data, and managing the inferencing phase in which employees interact with the model.

“That is when the company came up with an answer – private AI. It is an architectural approach, meaning it’s an approach that can be done by others as well, addressing the privacy issues while delivering exciting business cases,” Raghuram highlighted.

So, what is private AI?

Throughout the two-day Explore event in Singapore, there was a common understanding as to the issues VMware was addressing: generative AI is now making the privacy challenge both more consequential and more complex. “Data is the indispensable ‘fuel’ that powers AI innovation, and the job of keeping proprietary data private and protected has intensified,” Raghavan told reporters following his keynote.

READ NEXT

Safely using AI within the legal profession

VMware’s plan from the get-go was to design a new approach that balanced the tremendous business value of AI with privacy safeguards it could trust. Raghavan further shared how enterprises are being asked to address three key privacy issues: “First, how do you minimize the risk of intellectual property ‘leakage’ when employees interact with AI models? Second, how do you ensure that sensitive corporate data will not be shared externally?”

According to VMware’s CEO, the third issue enterprises want to address is how they can maintain complete control over access to their AI models. In contrast to public AI models, that can expose businesses to various risks, private AI is an architecture built from the ground up to give businesses greater control over how they select, train, and manage their AI models. “That level of control and transparency is precisely what every legal team is now demanding,” Raghuram emphasized.

VMware Private AI Foundation with Nvidia

During the first Explore event in Las Vegas in August, VMware unveiled private AI, which allows businesses to create generative AI in their data centers, rather than relying on the cloud. Private AI brings AI models to the locations where data is generated, processed, and utilized. Concurrently, VMware also disclosed a partnership with Nvidia, the leader in the AI chips sector. 

The VMware Private AI Foundation with Nvidia is intended for release in “early 2024.” It consists of a set of integrated AI tools that will allow enterprises to run proven models trained on their private data in a cost-efficient manner. These models will be deployable in data centers, on leading public clouds, and at the edge.

The partnership between the companies will allow enterprises access to proven libraries that are vetted by both VMware and Nvidia, along with the ability to better use the relatively small number of GPUs that are available, and finally protect corporate data and intellectual property when releasing trained models into production.

VMware Private AI Foundation with NVIDIA, extending the companies’ strategic partnership to ready enterprises that run VMware’s cloud infrastructure for the next era of generative AI.

What does the VMware-Nvidia collaboration mean?

In addition to Nvidia exploring a fresh path for diversification, this collaboration would be highly advantageous for VMware. The company stands to gain significantly as the partnership seeks to ensure compatibility with major companies such as Dell, Lenovo, HPE, and more, making it straightforward for customers to implement the solution per their specific needs. This could lead to widespread adoption across various industries.

Furthermore, Nvidia is equipping VMware with advanced computing capabilities. During his keynote address at VMware Explore Las Vegas, Nvidia’s CEO Jensen Huang unveiled a significant advancement that empowers VMware to achieve optimal hardware performance while preserving security, manageability, and the ability to migrate resources seamlessly across different GPUs and nodes.

Nvidia’s keynote at VMware Explore Las Vegas. Source: X

“GPUs are in every cloud and on-premise server everywhere. And VMware is everywhere. So, for the first time, enterprises worldwide will be able to do private AI. Private AI at scale deployed into your company is fully secure and multi-platform,” Jensen said. That will lead to an increased and optimized usage of Nvidia’s GPUs.

It could also relieve data center costs and address the GPU crunch during the ongoing AI gold rush. By harnessing the full potential of computing resources, encompassing GPUs, DPUs, and CPUs through virtual machines, the Private AI Foundation ensures optimal resource utilization, ultimately translating into reduced overall costs for enterprises.

Another critical advantage of the partnership is flexibility. “For any meaningful enterprise, their data lives in all types of locations. Distributed computing and multi-cloud will be at the foundation of AI; there’s no way to separate these two,” Raghuram explained.

The collaboration between VMware and Nvidia marks the emergence of on-premises generative AI, transforming the AI landscape and establishing a mutually beneficial partnership that brings advantages both to the companies and to the broader ecosystem.

The post VMware Explore: private AI is the next frontier for enterprises. appeared first on TechHQ.

What happens when a grand master of memory, a motorcycle-restoring expat, and a neuroscientist team up? In the case of Ed Cooke, Ben Whately, and Greg Detre, the answer is that the language-learning app Memrise gets built – a process that included driving a bus across Europe to film a video dictionary of native speakers in 10 countries.

To keep operations on the road and make sure that the language learning app hit its targets, the trio recruited Steve Toy – who’d gained COO experience working in the device software space – as CEO. Today, the application’s tech stack includes numerous AI integrations, such as the use of large language models (LLMs), as well as speech-to-text (STT) and text-to-speech (TTS) technologies.

Relatively inexpensive AI

Developers can sometimes have reservations about using generative AI based on the costs of GPU time, but Toy has a different view. He makes the point that it’s much more affordable to synthesize content rather than having to film everything from scratch, recalling the founders’ trip in the ‘Membus’ in the early days of the language learning app.

However, while the technology behind the user experience may have changed, the foundations can still be summarized in three words – learn, immerse, and communicate. “First, we teach you vocabulary that you’re actually going to use. Next, we show you that language in real-life contexts, through immersive videos. Finally, you use what you’ve learned to practice conversing confidently,” writes Memrise, describing its approach to language learning.

READ NEXT

Is running AI on CPUs making a comeback?

Digging a bit deeper into how the app works, users build up an ever-expanding dictionary of words, and experience how phrases are used and spoken by watching videos. And while this may just sound like a stock language learning app or online training experience, Toy emphasizes that Memrise does things differently. And he’s right.

Users have the chance to upload any video and turn that content into a lesson that’s customized for them. “Technology has kicked that door open,” Toy told TechHQ.

Thanks to AI tools working behind the scenes, users can select any scenario that appeals to them and understand how that plays out in a foreign language. Considering corporate training opportunities, the functionality could be a game-changer for firms that want to equip their staff with terminology in areas that matter to the business. And, in the future, Memrise could find itself supporting B2B services alongside its regular B2C revenue streams.

Toy explains that there’s more to the custom courses than just being a translation from one language to another. It’s important not to drop language learners straight in at the deep end, as users will grow frustrated if all of the words and phrases are completely alien to them.

Instead, the software compares the content with the user’s language learning app dictionary to find segments that are likely to be comprehensible, allowing proficiency to be built at steady intervals. And, again, you can see how this approach would work in a business environment, as well as for classic travel scenarios such as ordering food in a restaurant.

When it comes to practicing their skills, users have the option of interacting with an AI-powered chatbot. Conversational AI has made it possible to deliver realistic language training as never before. Chatbots can listen to spoken phrases, speak their replies, understand text, and offer words of help.

Steve Toy, CEO of Memrise. Image credit: Memrise.

This year, Memrise partnered with Discord to bring its language learning app chatbot to the popular instant messaging social platform. Discord members can converse with the advanced chatbot, which is supported by GPT-3, much like they would chat with their friends on the server. And users have two options /learn_solo and /learn_together.

Reviews of the Discord app are positive and describe the experience as a fun way to learn a language. It gives users the chance to run through a scenario ahead of carrying out the same task in the real world. And it’s easy to see how businesses could benefit from such a tool to help train their staff.

In the past, building such a chatbot would have consumed a lot of programming time, with developers having to craft numerous rules. All of the user interactions would need to be hardcoded. However, that all changed with the introduction of GPT-3 and subsequent LLMs.

Conversational AI algorithms based on next-word predicting neural networks are happy to chat about a wide variety of topics, given the vast size of the training datasets used by OpenAI and other foundation model creators. Plus, those conversations can take place in multiple languages, which is remarkable.

Edge computing options for language learning apps

The future for language learning apps certainly seems like a bright one, but there are still some issues to solve. Currently, Toy and his team are looking at edge-computing options to bring AI features closer to the device. He wants users to be able to develop their language skills on a plane, for example, when internet connectivity isn’t available.

The next step is to identify which part of the LLM is relevant to the language learning app and cleave that portion off to user devices. Many software developers are in a similar boat, relying on API calls to the cloud to provide AI-enabled features. In the future, LLMs that have been constrained to support particular types of queries and with a much smaller footprint, could pave the way for serving features locally.

READ NEXT

Airbnb for GPUs – decentralized IaaS for generative AI

On TechHQ we’ve written about how it’s possible to dynamically trim the number of artificial neurons that participate in model inference. Companies such as ThirdAI have shown how the approach can lift the performance of algorithms running on CPUs to levels typically only seen using GPUs. And it’s something that could benefit app makers with AI features such as Memrise.

Finally, it’s worth covering off the competition between Memrise and other language learning apps such as Duolingo or OpenAI-backed Speak. If you’re asking what’s the best language learning app, Memrise or Duolingo – to highlight a popular search query – then you could be missing out on the opportunity to use multiple resources.

The more practice, the better, and the reward of using a combination of language learning apps is that you get to fire more neurons (real ones, not just AI versions) more often. It’s a strategy that’ll help vocabulary, key phrases, and sentence structures to stick in the brain for when you need them to seal your next business deal.

The post Language learning app eyes up B2B and edge opportunities appeared first on TechHQ.

Artificial intelligence (AI) is transformative across today’s industrial landscape. Everything from enterprise software to machine automation is benefiting from the ability of multi-layered neural networks – with sufficient data and training – to make sense of our world. But as the size of deep learning models balloons, opening the door to more complex natural language processing (NLP) and other AI applications, so does the amount of compute that’s required. And that’s a problem when it comes to edge AI.

Edge AI trend

Deploying deep learning algorithms on portable computing hardware such as smartphones or onboard vehicles gives users access to powerful image recognition capabilities – to give just one of many use cases. And running models locally on edge AI hardware provides resilience against any interruption in connectivity.

There are also energy considerations. Users are starting to question the environmental impact of running giant AI algorithms in the cloud, given the energy cost of training models with billions of parameters and consuming large amounts of cooling water in the process. But, as it turns out, developers have become experts at pruning their models to reduce the computing demands of deep learning inference with only a minor impact on the accuracy of results.

READ NEXT

Is running AI on CPUs making a comeback?

These efficiency measures are great news for enabling edge AI. And to understand how the various methods work, it’s useful to first paint a picture of deep learning and consider how multi-layer neural networks turn inputs into meaningful outputs.

At an abstract level, you can think of a deep neural network as a universal function approximator. Given enough parameters, everything can be represented by a mathematical function. You might have seen formulae that look like shells when plotted in 3D or fractals that resemble tree branches. And large numbers of artificial neurons have proven to be capable of describing images and finding missing words in sentences.

Training these AI algorithms involves adjusting millions of model weights to make patterns of artificial neurons sensitive to certain inputs, such as edge features in an image. It’s also necessary to set biases for each of the nodes in the network, to determine the strength of the activation that’s required to make the corresponding artificial neurons ‘fire’.

If you’ve ever seen an analog music synthesizer covered in knobs, this is a good analogy, but multiply the number of dials by a million or more. And our input could be the feed from a video camera, which – after passing through all of the settings – turned on a light whenever a dog was seen in the image.

A Survey on Model Compression for Large Language Models

"Addressing the imperative need for efficient deployment, we delve into various methodologies, encompassing quantization, pruning, knowledge distillation, and more."https://t.co/aXv66pAdFv#LLM #papers pic.twitter.com/KAiOEcXTEz

— llm.ai (@llmgen_AI) August 22, 2023

Looking at the numbers on the dials, we might see that some parameters are more important than others. And that brings us to the concept of model pruning, which is one approach to squeezing algorithms onto edge AI hardware.

Today, developers use a variety of methods to make edge AI neural networks faster to run and smaller to accommodate without compromising performance. One approach is to zero out very small model weights, which can pinpoint artificial neurons that have little impact on how the algorithm behaves.

Another trick is to retrain the pruned model over a few iterations, which may result in fine tweaks to the other parameters, to recover any of the lost accuracy. Some pruned image recognition algorithms can behave more effectively than the original neural networks, which is a great result for edge AI.

Unfortunately, large language models (LLMs) can be harder to optimize as the retraining step isn’t trivial. But a new approach termed Wanda (pruning by weights and activations), which has been evaluated on the LLaMA family of LLMs, shows that considering activation paths allows 50% of the structure to be pruned without a major loss in performance. And, importantly, the training doesn’t need to be rerun to update the weights.

Considering how the weights are represented can help too – for example, storing values as 8-bit integers rather than in single-precision floating-point format (FP32) can save dramatically on memory. Conventionally, model weights are scaled to between zero and one, but those values can still be recovered from the memory-saving integers for processing.

Another strategy for making algorithms more efficient for edge AI applications is to deploy so-called teacher and student models, where the student learns from the richer information provided by the teacher. Specifically, the teacher model can give the student model the probability distribution of the most likely results as training inputs.

READ NEXT

What can generative AI do for business?

This approach has been used successfully to build DistilBERT, a distilled version of BERT: smaller, faster, cheaper, and lighter. Using teacher and student models (also known as knowledge distillation), Hugging Face researchers showed that it’s possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster.

And to understand why this is such a big deal, it’s worth noting that BERT is one of the most useful NLP models out there. BERT can be used for text encoding to retrieve similar passages from other data. It can summarize large amounts of text information and provide answers to questions.

Considering edge AI, lightweight NLP models could process data locally to preserve privacy and protect sensitive information that clients may not want to be sent to the cloud. And companies could use DistilBERT to build their own proprietary semantic search engines for navigating business data without having to send any of that data to Google, for example.

AI success stories

AI success stories in the cloud are inspiring a variety of use cases. And, as developers become more accomplished at compressing that algorithmic performance into smaller footprints, we can expect those benefits to translate into edge AI applications too.

Also, users have a growing number of tools to lean on to optimize their machine-learning models. Google’s TensorFlow Model Optimization Toolkit supports the deployment of models to edge devices that have restrictions on processing, memory, power consumption, network usage, and model storage space.

READ NEXT

Can zero-trust LLMs overcome poisoned generative AI?

There are other options too, such as model optimization SDKs that enable efficient deep learning for embedded systems. And providers include Swedish deep tech firm Embedl, which recently raised 45 MSEK (USD 4.1 million) to scale up its operations.

“Embedl’s solution provides significant performance enhancement when developing Autonomous Driving Systems (AD) and Advanced Driving Assistance Systems (ADAS) in the automotive sector,” writes the firm on its website. “It will also allow AI to be incorporated into consumer products with less powerful hardware.”

According to the company, customers can use the SDK to create deep learning algorithms that can be run on battery-powered devices, which signposts another trend in edge AI.

The post Edge AI: how to make deep learning more efficient appeared first on TechHQ.

From the early stages of the internet’s widespread adoption to the present era of hyperconnectivity, the volume of data we generate has grown exponentially. The proliferation of smartphones, the Internet of Things (IoT), and the seamless integration of technology into everyday life have all contributed to this increase. By 2025, we are expected to generate 463 exabytes of data each day.

For businesses, this surging digitalization has resulted in new customer expectations and necessitated innovative approaches to meet them. The largest industries, like financial services, retail, telco, and healthcare, face the challenge of effectively processing and utilizing the vast amounts of data they have access to. Traditional data processing methods are being challenged to keep up with the sheer scale, velocity, and variety of information businesses generate like never before.

Enterprises are turning to cutting-edge technologies such as GPU-based cloud computing, artificial intelligence (AI), and distributed edge computing to revolutionize their operations. Cloud computing provides scalable storage and computing resources, and machine learning (ML) techniques can be used to extract meaningful insights from complex datasets. By leveraging ML, businesses can automate processes, optimize decision-making, and deliver personalized customer experiences.

Edge computing enables data processing and computation to occur as close as possible to the data source, reducing the need for data to be transmitted to a distant data center. By bringing computing capabilities closer, an edge-focused solution can enable real-time processing with reduced latency and enhanced responsiveness for end-users.

The financial sector is already starting to embrace edge devices, as businesses can process data faster and use real-time analytics to offer personalised services. Processing data on these (typically) smaller devices also reduces transmission and infrastructure costs and allows the company to more easily meet local regulatory requirements.

However, other industries can also reap these same benefits. In retail, for example, edge computing can improve in-store customer experiences through faster data processing for personalized offers and real-time inventory management. In the healthcare sector, it can facilitate real-time diagnostics and decision-making for doctors, as well as support remote patient monitoring and telemedicine applications.

Source: Shutterstock

It has been predicted that over 50 percent of enterprise-managed data will be created and processed outside the data center or cloud by 2025. This will be partly thanks to several developments emerging from edge computing. One of these is the use of edge in combination with 5G. Together they reduce latency even further and support real-time processing capabilities creating massive opportunities for telco providers. Edge computing will also likely extend to complement cloud computing for large processing jobs rather than replace it altogether, providing burst capability, for example, at peak demand times. Finally, the edge will be more closely integrated with IoT hardware and local ML-powered applications to improve performance through more efficient data processing close to where IoT and IioT provide critical services.

However, embracing edge computing is not as straightforward as plugging in a new device. It is critical to protect the IT equipment within the devices. There are specific challenges associated with distributing compute in places where IT is not meant to reside – heat, humidity, dust and other environmental debris that can damage the equipment. Exposure to contaminents can cause failure of edge devices and result in reduced performance, shorter hardware lifespans, and costly device damage. Fluctuations in temperature can also making cooling the equipment challenging and can throttle performance. Having purpose built equipment that is sealed, reinforced and ruggedized is critical to distributing compute in new locations.

As in a centralized data centre facility, edge solutions also need to contend with heat generation and energy consumption during data processing, so cooling becomes critical to ensure the devices’ optimal performance, longevity, and reliability. Cooling technology from data centers may not be appropriate for edge installations, and, as many businesses have their own sustainability goals to meet, having an efficient solution that cools and protects IT equipment is crucial to minimize the environmental impact of edge computing operations.

Tank immersion, where the edge device is completely submerged in a non-conductive dielectric liquid, can address some of these concerns. However, this often requires a significant amount of space, large initial set-up costs, and more complex device servicing. Precision Liquid Cooling, where  a small amount of dielectric coolant is precisely targeted to remove heat from the hottest components of the server, has been developed to address all of these concerns. Through these systems, the heat from the hotspots, like the CPU and memory components, is captured and removed with an in-built heat exchanger.

Precision Liquid Cooling is more scalable than tank immersion. Offered in a standard vertical rack form factor, it offers a finer level of control over cooling distribution and can be tailored to specific components, reducing the risk of thermal bottlenecks. Similarly, it is also a more serviceable solution, as individual units can be easily accessed and maintained without disrupting the entire cooling infrastructure. This serviceability not only reduces downtime during maintenance but also lowers operational costs by enabling hotswapping of devices and allowing targeted repairs or upgrades to specific components as needed. According to Iceotope Technologies, a leading provider of Precision Liquid Cooling systems, the technology reduces component failures by 30 percent and extends the edge server’s lifespan.

The cooling solution is significantly more sustainable than many cooling alternatives. While Precision Liquid Cooling removes nearly 100 percent of server heat, it requires 40 percent less energy and reduces carbon emissions by the same amount. In fact, Precision Liquid Cooling offers approximately 6x power density improvement per square meter resulting in significantly reduced carbon footprint. Water consumption is also minimal due to the absence of mechanical chilling requirements, and significantly less dielectric fluid is required than in-tank immersion solutions.

A new solution is necessary for this new era of heightened data demands. Discover how Iceotope’s Precision Liquid Cooling technologies can enable your business to harness the power of edge computing for enhanced business growth and productivity today, while reducing costs, maximising revenue and accelerating sustainability objectives.

The post Staying cool under data pressure: The role of Precision Liquid Cooling in edge computing appeared first on TechHQ.

• There is no “one language to rule them all.”
• Various computer programming languages have been designed to solve particular problems.
• That’s only likely to happen more often in the era of gnerative AI and quantum computing.

That there are so many computer programming languages is widely known. What isn’t so easily understood is why – coding is already complex enough; shouldn’t developers be able to read from the same book?

Since the dawn of computer programming there have been language variants. Is it simply because, as one Reddit user says, “developers LOVE to reinvent the wheel”?

One computer programming language for all

Attempts have been made to develop a coding language that could be used universally. In 1964, PL/I was designed by IBM, with no intention of ever needing, say, a PL/II. Back when programming used hole-punched cards, One Good Programming Language would simplify the development process as well.

Work on the PL/I specification began in 1964 and work on the first compiler began in 1966. Theoretically, PL/I would bring together the power of three different programming languages:

• FORTRAN – The scientific programming language (1954)
• COBOL – The business programming language (1959)
• ALGOL – Primarily a research language, but with innovative paradigms and features (1958)

(Note: in the early days of computing, programming languages could be quite simply named for what they did: FORTRAN = formula translator, COBOL = Common Business Oriented Language, ALGOL = Algorithmic language.)

This seemed reasonable enough. Combining three to make one didn’t appear to be the mammoth task it turned out to be:

Source: Communications of the ACM

This theoretical forgets that establishing PL/I wouldn’t be just a development effort, but in effect a system conversion. Yes, the new language would require developers to begin using a new language, but, and this might sound obvious, they’d have to stop using FORTRAN, COBOL, and ALGOL. This in turn would mean converting their existing codebases and solutions to PL/I.

In short, there was more to it than was initially anticipated. Compounding this problem was the fact that the three languages that PL/I set out to unite were themselves evolving in real time. A system conversion was, realistically, untenable.

READ NEXT

Tic-tac-toe superposition entangles quantum advantage

Despite failing to achieve its principal aim, PL/I was a relatively successful computer programming language. It was taught at college level and reportedly was used in the development of the Multics operating system and the S/360 version of the Sabre airline reservation system, among others.

A toolbox of computer programming languages

So, is a failure to implement a universal computer programming language early on the reason why there isn’t one today? Aside from anything else, a good developer is not only productive but able to use language features in creative ways. If there are edge-cases to be found, developers will find them – especially at scale.

Pushing a language to its extremes means uses that the original authors won’t have foreseen. That means issues arise, are solved and thus, a new language is born. A current programming language might not deal with the specific issues that arise as a result of the nature of the language’s capabilities. So, just as technology evolves, the languages it runs on do too.

FORTRAN, created by John Backus, is the oldest programming language still in use today.

If we really split hairs, the first programming language was written on a piece of paper by Ada Lovelace in 1843. About a century later, Konrad Zuse developed the first ‘real’ programming language called Plankalkül (Plan Calculus). Among other things, that language enabled procedures to be created, which stored chunks of code that could be invoked over and over to perform routine operations.

YOU MIGHT LIKE

HARDWARE

No coding required? Alphabet working on self-learning robots

Just because new languages are born as solutions to a specific issue though, doesn’t mean they aren’t more widely applicable. Some notable examples of programming languages that developed from specific needs include:

1. C – Dennis Ritchie and Bell Labs created C in 1972 to overcome the problems of older languages. They first used it to run the Unix operating system. Today, it can run on almost all kinds of software and hardware.
2. C++ – Bjarne Stroustrup created C++ in 1985 for two main reasons. The first reason was to upgrade the features of the C programming language, and the second reason was to make it accessible for everyone to use in all fields.
3. PHP – PHP was created in 1994 by Rasmus Lerdorf, and he initially used it for a personal project. Later he improved the language and released it to the public, and developers started using it heavily for web development.
4. Go – Google created Go in 2009 because the languages it worked with at the time took were too slow and inconvenient for certain tasks. Also, Google wanted a programming language that was fast in performance but still relatively simple to read.
5. JavaScript – Netscape created JavaScript in 1995. It initially created it for front end development, so both web developers and designers could learn and create website features in a short timeframe. Now we use it for back end, front end, and mobile development.

Javascript and PHP, along with Python and Perl, are derivatives of C. Swift, developed by Apple in 2014, was a replacement for C, C++, and Objective-C, intended to be easier to use and allow less room for error. Swift’s versatility means it can be used for desktop, mobile, and cloud applications.

Languages are born, evolve, and go extinct in the computing world.

A fun one: Rust began as a personal problem. In 2006, Graydon Hoare was a 29-year-old programmer working for Mozilla. This meant that when the elevator in his apartment building – where he lived on the 21^st floor – kept breaking, he knew it was due to crashes caused by problems with how a program uses memory.

READ NEXT

‘Coding is for everyone’ — thoughts from an Ocado engineer

The software in elevators and similar devices is written in C++ or C languages, which famously allow programmers to write code very fast, but also make it easy to accidentally introduce memory bugs.

Hoare set about writing a new computer language that he hoped would make it possible to write small, fast code without memory bugs. He named it Rust after a group of fungi that he says are “over-engineered for survival.”

By the start of 2023, Rust was being used by 2.3 million coders.

The evolution of computer programming languages visualized, much like a family tree. Source: Computer History Museum.

Ultimately, asking why there are so many computer programming languages is like asking why a handyman needs a full toolbox; coders need different things depending on what they’re setting out to achieve.

Sure, the corkscrew on your Swiss Army Knife works, but wouldn’t you choose to use a purpose-built, ergonomic one if you had the choice?

If the Eighties taught us anything, it was that you needed a different programming language to blow up the world than you did to improve your grades or take a day off.

The post Why there’s no universal programming language appeared first on TechHQ.

Cloud computing has become so popular – enabling users to deploy IT resources affordably at the click of a button – that the idea of building out those services locally may feel like an odd choice. But there are still good reasons for bringing hardware closer to home, such as eliminating latency effects, reducing bandwidth constraints, and having full control over your data. And IT security teams will need to modify their defense strategies depending on whether information is held locally or hosted in the cloud, and that includes keeping on top of edge computing security issues.

The fog between the edge and the cloud

One of the biggest contributors to the rise of edge computing, where a significant portion of data processing is carried out locally rather than in the cloud, is the now mature internet-of-things (IoT) sector. It’s estimated that there are billions of IoT devices in use globally, and while – by definition – these nodes are networked over the web, they still carry onboard electronics to gather information locally and make decisions based on that data.

READ NEXT

Smart tech for the 15-minute city

Also, processing outcomes can be the result of local inputs and analytics performed in the cloud. And this fusion – sometimes dubbed fog computing – couples the speed of having devices locally on the edge of the network with the advantages of being able to lean on large amounts of processing power and data insights available centrally.

From an edge computing security perspective, decentralized fog architectures can have benefits that go beyond pure data processing and decision-making performance. Unlike a pure-play cloud application where everything is processed centrally with large amounts of web traffic, fog computing designs can be constructed to minimize exposure over the internet. More sensitive information can be retained on the local network, raising defenses against remote attackers.

Also, fog nodes can carry out pre-processing and buffering to make cloud execution of tasks more efficient. Rather than send everything out to the web, increasingly – for a variety of reasons, including speed and privacy – edge computing devices tap into the internet only when necessary.

Addressing physical security issues of edge computing

So far, so good, in terms of reducing the target size of valuable information that could be exposed over the web. But that data privacy gain has shifted security issues to the edge computing setup. Thankfully, it’s not straightforward for an attacker to walk into a cloud services provider’s facility and start tampering with devices. But edge computing architecture could be located on the roadside – in the case of smart cities and connected and autonomous vehicle applications – where physical security issues could be more pressing.

In many cases, putting edge computing equipment up high on the roof of a nearby building will address many of those physical security issues. And the equipment can benefit from the access controls provided by the facility’s front desk, camera surveillance, and security guard patrols. But what if there aren’t buildings nearby? Or if facilities owners are unwilling to share their space?

READ NEXT

Self-driving skillset – game theory for autonomous vehicles

To avoid security issues, operators of edge computing equipment need to think much more carefully about device basics such as operating systems and boot sequences. Bad actors – with the advantage of physical access – will look to exploit known hardware vulnerabilities and have more metaphorical levers to pull compared with targeting cloud infrastructure.

Edge computing operators can use device attestation mechanisms to warn of equipment tampering or to check that a device requesting to join the network is legitimate. Device attestation can check not just hardware integrity, but also perform software validation and confirm that everything has been configured as expected. And it’s become a popular security tool for edge computing operators.

In terms of blueprints that developers can follow to raise defenses around their edge computing infrastructure, the evolution of mobile phone security provides a useful playbook for IT teams. And, in fact, by using those chipsets and associated security solutions, edge hardware developers can readily leverage that experience to bolster the defenses of their designs.

Threat scenarios to consider include thieves walking off with the equipment. And for that reason, it’s important that all data is encrypted not just in transit, but also at rest – at a bare minimum. Considering other protective steps, many IoT-focused operating systems now give developers fine-grained control over what processes are allowed to run, locking systems down to a much higher degree compared with fully-featured installations.

Benefits of a growing edge computing ecosystem

Also, as edge computing infrastructure proliferates through the continued adoption of IoT, which includes a wide range of applications in agriculture, manufacturing, healthcare, smart cities, and many other use cases, so does the understanding on how to keep systems safe.

Looking ahead, computing infrastructure is becoming more mobile. The complexity and capability of smartphones increases with every model update, augmented by companion devices. There’s also robotics to consider. Cobots – collaborative robots – are anticipated to become much more popular outside of their typical industrial settings. And activity is ramping up on the use of automated drones capable of flying unaided – with air corridors being proposed to support supply chain innovation and overcome bottlenecks on the ground.

“It’s important that we acknowledge that the edge computing world is growing,” said Rob High – VP & CTO of Edge Computing at IBM. “It will have as much impact in the world of enterprise computing as mobile phones did in the world of consumer computing.”

The post Edge computing security issues: mitigating local IT threats appeared first on TechHQ.

Edge computing (data processing on the peripheries of a network, as close to the data source as possible) is a technological approach that has been growing in popularity in business circles over the course of the last decade. But it has really begun to go mainstream since the emergence of unstructured data as a force to be reckoned with. And while the economic future may be uncertain, one thing looks distinctly likely: edge computing is set to make a much more significant mark in 2023. In fact, by 2023, it’s estimated that more than 50% of new enterprise IT infrastructures will be deployed at the edge.

READ NEXT

Edge computing keeps 5G in the fast lane and on the pulse

But why? And perhaps more importantly, how is edge computing likely to impact the way businesses operate in 2023? There are a handful of edge computing trends that seem destined to become daily realities across the next 12 months.

A tsunami of data

Did we mention unstructured data? That has become a real issue – all the data that doesn’t fit neatly into a database has to be found, dealt with, and organized somehow, and in particular, that rise in unstructured data has been linked with the rise and rise of Internet of Things (IoT) devices.

Forecasters say we could be looking at a worldwide generation – generation on top of all the existing data – of around 175 to 180 zettabytes (that’s 175 trillion gigabytes in old money) of data per year across 2023, 2024, and 2025. All that data, both structured and unstructured, particularly from IoT devices, will need processing, and the logical way to deal with IoT data is on the edge, rather than pulling it into the central system for processing and mining for profitable insights.

READ NEXT

Edge computing platforms celebrate on-site wins

It’s worth putting the rise of IoT devices into perspective. By the end of 2022, there are expected to be some 13.1 billion IoT connected devices in the world. By the end of 2023, that number is expected to be around 15.1 billion.

There are a little under 8 billion human beings on the planet right now, and by no means all of them have even one IoT device. You should feel free to do the math.

IoT devices represent a data tsunami of staggering proportions, and the best – in fact, the only effective – way of mining those devices for data insights is on the edge.

And there are only going to be more of them in 2023.

The speed of 5G

With the exception of people who believe 5G fries your brain or causes Covid, there’s a single, simple fact that everyone understands about 5G – it means faster data processing.

Having finally emerged from the deadly Covid era, many big technology nations, including both the US and China, have emerged with powerful 5G networks. That means more edge computing will be possible closer to the data source than ever before. How that breaks down in terms of individual companies and sectors is yet to be discovered – even by those companies and sectors.

But the ability to deliver significantly more edge computing at speed is likely to act as the keys to a whole new kingdom of business functionality for the technology. If you haven’t started thinking yet about how your business could benefit from edge computing and 5G – your competitors may very well have.

Start thinking about how to maximize the effectiveness of the technology combination right now – from rapid data collection to improving digital experiences at a distance with super-low latency – to make the most of 2023’s opportunities.

Improved agri-tech

Agri-tech has been growing in importance for years, but 2023 is likely to bring a whole new dimension to what is possibly on farms, both large and small. The logic is obvious – even small farms are large spaces, frequently spread out. Better edge computing means you can intensify the efficiency of your agri-tech, processing data at a distance and delivering better, more economical farming outcomes.

Faster machine learning on-site

Edge computing allows machine learning at remote locations to be feasible in the sort of timeframes previously only available to non-ML applications. If 2023 is the year when edge computing becomes relatively ubiquitous at last, it will normalize machine learning at speed and at a distance, allowing staff and customers to trust in the uses and results of machine learning and AI in ways they previously wouldn’t have, by cutting down the latency in the process to acceptable levels.

READ NEXT

A CXOs guide to fog, cloud and edge computing

The non-cloud cloud

With inflationary pressures bearing down on every business, and cloud storage becoming increasingly expensive (especially given the need to process unstructured data, often across multicloud environments), the drive to find less expensive alternatives to the cloud will be stronger in 2023 that it has been until now. Using edge computing keeps data processing away from central infrastructure silos, and so it can help defer the costs of otherwise necessary cloud storage.

The year on the edge

Whether your company is heavily focused on customer data capture, or whether the bonus of hugely reduced latency through the 5G-edge computing combination will lighten the financial burdens on your company, 2023 is likely to bring you significant business benefits – either speeding up what you already do, or helping you expand the range of what you can do on the edge that’s new, and what you can do on the edge that’s more than you’ve ever done before.

The post Edge computing trends in 2023 appeared first on TechHQ.

When it comes to big trends in tech, voice-enabled applications definitely place high on the list. Companies have made great progress in using voice-tech as an authentication tool. In healthcare, speech recognition services are streamlining workflows and saving clinicians hours of administration. And a fascinating third piece of the puzzle is the rise of voice AI for creating advanced and highly customizable digital assistants – a technology that has its origins in query-by-singing/humming software for smartphones.

Voice-based future

“When we came out of stealth mode in 2015, we knew that our technology could do so much more,” James Hom, Chief Product Officer and co-founder of SoundHound told TechHQ. “We’ve always had a vision that people could talk to their devices and have spent a lot of time making sure that our models are robust.” Today, much like as Hom predicted, SoundHound’s technology has become a platform for a wide range of voice-enabled applications.

In automotive, software engineers are working with OEMs to develop much more accurate voice assistants that can provide information to drivers and other passengers based on natural speech queries. And clients include Honda, Hyundai, Kia, Mercedes, and Stellantis, which owns Alfa Romeo, Chrysler, Jeep, and Maserati, to name just a few brands in the group. Using voice to enter destinations and query navigation details means that drivers can keep their hands on the wheel and eyes on the road ahead. “Voice in automotive is a no-brainer,” said Hom.

READ NEXT

Powerful AI models are healthy for voice tech

Typically, voice-enabled applications rely on a two-step operation to make sense of spoken queries. The first – automatic speech recognition – involves comparing a phoneme sequence against a pronunciation dictionary. Then, in the second part, natural language processing is used to define meaning from the spoken words. But this process takes time and doesn’t always provide the results that users are expecting. To improve on this, SoundHound’s team instead applies neural network architecture to map input sequence data (the voice query) into output sequence data (a structured request). The method provides an efficient way of separating the intent from the variables in a spoken query – for example, the intent could be ‘tell me the weather’ with the variable ‘in London’.

Acoustic upgrade

There are other advantages too, such as the ability to tune the model so that it can perform well not just in a quiet setting, where voice commands can be clearly understood, but in noisy environments too. Engineers can use pilot data to compensate for the audio characteristics of different vehicles, for example. It also means that the technology is well suited for food ordering systems deployed in busy restaurants – another application that’s on the rise for voice AI.

Hom points out that use cases can even be combined – for example, a driver could ask the in-vehicle voice assistant to find restaurants nearby and then, after selecting one, choose items off the menu by piping the audio through to the automated food ordering system at the end-destination. Smart TVs are another application where voice AI is giving customers a whole new level of integration, as well as greater insight into user behaviour.

READ NEXT

Edge computing keeps 5G in the fast lane and on the pulse

“One of the things that we are proud of is that we partner with the people we work with,” said Hom. And this includes giving clients access to their data. SoundHound can provide dashboards that help customers to identify features that users are asking their voice assistants for. Another popular selling point of the technology is the ability for companies to develop a branded voice. “We can train our product completely from scratch,” adds Hom. “There are lots of options, including using machine learning to give more natural sounds.”

Embedded approach

Voice AI has taken great strides in solving the problems that made progress hard in the early days of speech recognition. There’s a big difference between asking for ‘no mayo’ versus ‘extra mayo’, for example, and models needed to reach a point where they could capture key details reliably. There have been advances in other areas too such as embedded solutions, which have opened the door to voice assistants being available on the edge.

Packaging voice AI as a standalone chipset means that systems can operate in scenarios where connection to the internet could be intermittent. Such configurations offer continuity of service – for example, in automotive applications as vehicles pass through a tunnel. What’s more, active arbitration schemes mean that live data, such as weather or sports results can be gathered when cloud connectivity becomes available. Another benefit of edge solutions is that developers have the option of building solutions that are completely cloud-independent, which may be useful in manufacturing or healthcare settings, for example. “The availability of more edge options will allow businesses to store and protect sensitive data locally – which could help brands build customer trust,” commented Hom.

Bringing clients on board also involves making solutions available in multiple languages. “The core engine is language agnostic, but there is a language-dependent part, which is where our team of linguists and other experts fits in,” said Hom. In 2022, SoundHound’s platform supports 25 languages and the firm is aiming to add over 100 languages and variations as part of future developments.

The post Digital assistants: neural networks drive voice AI growth appeared first on TechHQ.