All posts by Richy George

How knowledge graphs improve generative AI

Posted by on 9 October, 2023

This post was originally published on this site

The initial surge of excitement and apprehension surrounding ChatGPT is waning. The problem is, where does that leave the enterprise? Is this a passing trend that can safely be ignored or a powerful tool that needs to be embraced? And if the latter, what’s the most secure approach to its adoption?

ChatGPT, a form of generative AI, represents just a single manifestation of the broader concept of large language models (LLMs). LLMs are an important technology that’s here to stay, but they’re not a plug-and-play solution for your business processes. Achieving benefits from them requires some work on your part.

This is because, despite the immense potential of LLMs, they come with a range of challenges. These challenges include issues such as hallucinations, the high costs associated with training and scaling, the complexity of addressing and updating them, their inherent inconsistency, the difficulty of conducting audits and providing explanations, and the predominance of English language content.

There are also other factors like the fact that LLMs are poor at reasoning and need careful prompting for correct answers. All of these issues can be minimized by supporting your new internal corpus-based LLM by a knowledge graph.

The power of knowledge graphs

A knowledge graph is an information-rich structure that provides a view of entities and how they interrelate. For example, Rishi Sunak holds the office of prime minister of the UK. Rishi Sunak and the UK are entities, and holding the office of prime minister is how they relate. We can express these identities and relationships as a network of assertable facts with a graph of what we know.

Having built a knowledge graph, you not only can query it for patterns, such as “Who are the members of Rishi Sunak’s cabinet,” but you can also compute over the graph using graph algorithms and graph data science. With this additional tooling, you can ask sophisticated questions about the nature of the whole graph of many billions of elements, not just a subgraph. Now you can ask questions like “Who are the members of the Sunak government not in the cabinet who wield the most influence?”

Expressing these relationships as a graph can uncover facts that were previously obscured and lead to valuable insights. You can even generate embeddings from this graph (encompassing both its data and its structure) that can be used in machine learning pipelines or as an integration point to LLMs.

Using knowledge graphs with large language models

But a knowledge graph is only half the story. LLMs are the other half, and we need to understand how to make these work together. We see four patterns emerging:

  1. Use an LLM to create a knowledge graph.
  2. Use a knowledge graph to train an LLM.
  3. Use a knowledge graph on the interaction path with an LLM to enrich queries and responses.
  4. Use knowledge graphs to create better models.

In the first pattern we use the natural language processing features of LLMs to process a huge corpus of text data (e.g. from the web or journals). We then ask the LLM (which is opaque) to produce a knowledge graph (which is transparent). The knowledge graph can be inspected, QA’d, and curated. Importantly for regulated industries like pharmaceuticals, the knowledge graph is explicit and deterministic about its answers in a way that LLMs are not.

In the second pattern we do the opposite. Instead of training LLMs on a large general corpus, we train them exclusively on our existing knowledge graph. Now we can build chatbots that are very skilled with respect to our products and services and that answer without hallucination.

In the third pattern we intercept messages going to and from the LLM and enrich them with data from our knowledge graph. For example, “Show me the latest five films with actors I like” cannot be answered by the LLM alone, but it can be enriched by exploring a movie knowledge graph for popular films and their actors that can then be used to enrich the prompt given to the LLM. Similarly, on the way back from the LLM, we can take embeddings and resolve them against the knowledge graph to provide deeper insight to the caller.

The fourth pattern is about making better AIs with knowledge graphs. Here interesting research from Yejen Choi at the University of Washington shows the best way forward. In her team’s work, an LLM is enriched by a secondary, smaller AI called a “critic.” This AI looks for reasoning errors in the responses of the LLM, and in doing so creates a knowledge graph for downstream consumption by another training process that creates a “student” model. The student model is smaller and more accurate than the original LLM on many benchmarks because it never learns factual inaccuracies or inconsistent answers to questions.

Understanding Earth’s biodiversity using knowledge graphs

It’s important to remind ourselves of why we are doing this work with ChatGPT-like tools. Using generative AI can help knowledge workers and specialists to execute natural language queries they want answered without having to understand and interpret a query language or build multi-layered APIs. This has the potential to increase efficiency and allow employees to focus their time and energy on more pertinent tasks.

Take Basecamp Research, a UK-based biotech firm that is mapping Earth’s biodiversity and trying to ethically support bringing new solutions from nature into the market. To do so it has built the planet’s largest natural biodiversity knowledge graph, BaseGraph, which has more than four billion relationships.

The dataset is feeding a lot of other innovative projects. One is protein design, where the team is utilizing a large language model fronted by a ChatGPT-style model for enzyme sequence generation called ZymCtrl. Purpose-built for generative AI, Basecamp is now wrapping increasingly more LLMs around its entire knowledge graph. The firm is upgrading BaseGraph to a fully LLM-augmented knowledge graph in just the way I’ve been describing.

Making complex content more findable, accessible, and explainable

Pioneering as Basecamp Research’s work is, it’s not alone in exploring the LLM-knowledge graph combination. A household-name global energy company is using knowledge graphs with ChatGPT in the cloud for its enterprise knowledge hub. The next step is to deliver generative AI-powered cognitive services to thousands of employees across its legal, engineering, and other departments.

To take one more example, a global publisher is readying a generative AI tool trained on knowledge graphs that will make a huge wealth of complex academic content more findable, accessible, and explainable to research customers using pure natural language.

What’s noteworthy about this latter project is that it aligns perfectly with our earlier discussion: translating hugely complex ideas into accessible, intuitive, real-world language, enabling interactions and collaborations. In doing so, it empowers us to tackle substantial challenges with precision, and in ways that people trust.

It’s becoming increasingly clear that by training an LLM on a knowledge graph’s curated, high-quality, structured data, the gamut of challenges associated with ChatGPT will be addressed, and the prizes you are seeking from generative AI will be easier to realize. A June Gartner report, AI Design Patterns for Knowledge Graphs and Generative AI, underscores this notion, emphasizing that knowledge graphs offer an ideal partner to an LLM, where high levels of accuracy and correctness are a requirement.

Seems like a marriage made in heaven to me. What about you?

Jim Webber is chief scientist at graph database and analytics leader Neo4j and co-author of Graph Databases (1st and 2nd editions, O’Reilly), Graph Databases for Dummies (Wiley), and Building Knowledge Graphs (O’Reilly).

Generative AI Insights provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss the challenges and opportunities of generative artificial intelligence. The selection is wide-ranging, from technology deep dives to case studies to expert opinion, but also subjective, based on our judgment of which topics and treatments will best serve InfoWorld’s technically sophisticated audience. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Contact

Next read this:

Posted Under: Database
MongoDB adds generative AI features to boost developer productivity

Posted by on 27 September, 2023

This post was originally published on this site

After adding vector search to its NoSQL Atlas database-as-a-service (DBaaS) in June, MongoDB is adding new generative AI features to a few tools in order to further boost developer productivity.

The new features have been added to MongoDB’s Relational Migrator, Compass, Atlas Charts tools, and its Documentation interface.

In its Documentation interface, MongoDB is adding an AI-powered chatbot that will allow developers to ask questions and receive answers about MongoDB’s products and services, in addition to providing troubleshooting support during software development.

The chatbot inside MongoDB Documentation—which has been made generally available — is an open source project that uses MongoDB Atlas Vector Search for AI-powered information retrieval of curated data to answer questions with context, MongoDB said.

Developers would be able to use the project code to build and deploy their own version of chatbots for a variety of use cases.

In order to accelerate application modernization MongoDB has integrated AI capabilities — such as intelligent data schema and code recommendations — into its Relational Migrator.

The Relational Migration can automatically convert SQL queries and stored procedures in legacy applications to MongoDB Query API syntax, the company said, adding that the automatic conversion feature eliminates the need for developers to have knowledge of MongoDB syntax.

Further, the company is adding a natural language processing capability to MongoDB Compass, which is an interface for querying, aggregating, and analyzing data stored in MongoDB.

The natural language prompt included in Compass has the ability to generate executable MongoDB Query API syntax, the company said.

A similar natural language capability has also been added to MongoDB Atlas Charts, which is a data visualization tool that allows developers to easily create, share, and embed visualizations using data stored in MongoDB Atlas.

“With new AI-powered capabilities, developers can build data visualizations, create graphics, and generate dashboards within MongoDB Atlas Charts using natural language,” the company said in a statement.

The new AI-powered features in MongoBD Relational Migrator, MongoDB Compass, and MongoDB Atlas Charts are currently in preview.

In addition to these updates, the company has also released a new set of capabilities to help developers deploy MongoDB at the edge in order to garner more real-time data and help build AI-powered applications at the edge location.

Dubbed MongoDB Atlas for the Edge, these capabilities will allow enterprises to run applications on MongoDB using a wide variety of infrastructure, including self-managed on-premises servers and edge infrastructure managed by major cloud providers, including Amazon Web Services (AWS), Google Cloud, and Microsoft Azure, the company said.

Next read this:

Posted Under: Database
Is a serverless database right for your workload?

Posted by on 26 September, 2023

This post was originally published on this site

The serverless database continues to gain traction across the industry, generating a lot of hype along the way. And why not?

The idea that an application developer starting on a new project can provision a database, not worry about sizing compute and storage, or fine-tuning database configurations, and really only needs a general sense of workload pattern and transaction volume to approximate cost is a very enticing proposition. Plus, some might even see very strong potential to reduce the TCO of the database system.

As the theme around cloud cost management continues to percolate, the elastic pay-as-you-go model makes serverless even more attractive—if the app and customers behave.

A one-size-fits-some model

The serverless database model can be a great solution for unpredictable, spiky workloads. It can even be a fit for predictable, but not constant workloads, e.g., ecommerce on holiday weekends. It’s ideal for the application development team that may not have deep database tuning expertise, or may not quite understand their app usage patterns yet. Or, for teams that may prioritize availability and performance and care less about control of their database system and aggressive margin optimization.

I do not mean to suggest that serveless is a runaway budget-burning machine. Paying strictly for what you consume has huge potential to keep costs down and avoid waste, but it also requires you to understand how your application behaves and how users interact with it. Huge spikes in workload, especially those that the application should have been more efficient with, can also be very costly. You’re paying for what you use, but you may not always use what you expected to.

Regulatory considerations and limitations

As you consider a serverless database option, you will also want to consider where your organization or company policy lies in a shared responsibility model. For example, serverless is not going to be entirely suitable for highly regulated workloads with strong governance policies over database configuration changes.

To really reap the benefits of a serverless database means accepting that you are relinquishing even more control of your database to the provider than you would with even a traditional database-as-a-service (DBaaS) solution, which those highly regulated industries would not necessarily accommodate without the same strict governance policies they have in place today.

Tuning for workloads and cost management

Serverless is not just about scaling system resources instantaneously. It’s also about ensuring the database is properly tuned for the type of workloads it is processing to honor those guarantees to the customer, while also optimizing utilization rates of the system resources it is consuming.

While the consumer may not have to worry about managing the cost of cloud infrastructure in a serverless model, the provider still does, and it is in their best interest to ensure that the database system is optimally tuned for their customer’s workloads, and that the underlying systems are at optimal utilization rates.

It also means the provider is going to leverage multi-tenancy models wherever possible to pack as many database clusters on their servers as possible. This maximizes utilization rates and optimizes margins—for the provider. In a multi-tenant architecture, you must be sure that your provider will have the level of predictability in your workload, along with any number of other customers on those same servers. This would ensure there’s enough idle resources available to meet any increases in workload, especially the unpredictable ones. 

Ultimately, serverless databases are a great technology, but one that is still in its infancy and that presents challenges for both the consumer and the provider. A serverless database should not be viewed as the end all be all. It is a very powerful option that app teams should consider when selecting their database solutions among numerous other options.

Jozef de Vries is chief product engineering officer at EDB.

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to

Next read this:

Posted Under: Database
Oracle CloudWorld 2023: 6 key takeaways from the big annual event

Posted by on 22 September, 2023

This post was originally published on this site

In line with Oracle co-founder CTO Larry Ellison’s notion that generative AI is one of the most important technological innovations ever, the company at its annual CloudWorld conference released a range of products and updates centered around the next generation of artificial intelligence.

The last few months have witnessed rival technology vendors, such as AWS, Google, Microsoft, Salesforce and IBM, adopting a similar strategy, under which each of them integrated generative AI into their offerings or released new offerings to support generative AI use cases.

Oracle, which posted its first quarter earnings for fiscal year 2024 last week, has been betting heavily on high demand from enterprises, driven by generative AI related workloads, to boost revenue in upcoming quarters as enterprises look to adopt the technology for productivity and efficiency.

In order to cater to this demand, the company has introduced products based on  its three-tier generative AI strategy. Here are some key takeaways:

Oracle has taken the covers off its new API-led generative AI service, which is a managed service that will allow enterprises to integrate large language model (LLM) interfaces in their applications via an API. The API-led service is also designed in a manner that allows enterprises to refine Cohere’s LLMs using their own data to enable more accurate results via a process dubbed Retrieval Augmented Generation (RAG).

It has also updated several AI-based offerings, including the Oracle Digital Assistant, OCI Language Healthcare NLP, OCI Language Document Translation, OCI Vision, OCI Speech, and OCI Data Science.

Oracle is updating its Database 23c offering with a bundle of features dubbed AI Vector Search. These features and capabilities include a new vector data type, vector indexes, and vector search SQL operators that enable the Oracle Database to store the semantic content of documents, images, and other unstructured data as vectors, and use these to run fast similarity queries.

The addition of vector search capabilities to Database 23c will allow enterprises to add an LLM-based natural language interface inside applications built on the Oracle Database and its Autonomous Database.

Other updates to Oracle’s database offerings include the general availability of Database 23c, the next generation of Exadata Exascale, and updates to its Autonomous Database service and GoldenGate 23c.

In order to allow enterprises to operate its data analytics cloud service, dubbed MySQL HeatWave, the company has added a new Vector Store along with some generative AI features.

The new Vector Store, which is also in private preview, can ingest documents in a variety of formats and store them as embeddings generated via an encoder model in order to process queries faster, the company said, adding that the generative AI features added include a large language model-driven interface that allows enterprise users to interact with different aspects of the service — including searching for different files — in natural language.

Other updates to the service include updates to AutoML and MySQL Autopilot components within the service along with support for JavaScript and a bulk ingest feature.

Nearly all of Oracle’s Fusion Cloud suites — including Cloud Customer Experience (CX), Human Capital Management (HCM), Enterprise Resource Planning (ERP), and Supply Chain Management (SCM) — have been updated with the company’s Oracle Cloud Infrastructure (OCI) generative AI service.

For healthcare providers, Oracle will offer a version of its generative AI-powered assistant, which is based on OCI generative AI service, called Oracle Clinical Digital Assistant.

Oracle has updated several applications within its various Fusion Cloud suites in order to align them toward supporting use cases for its healthcare enterprise customers. These updates, which include changes to multiple applications within ERP, HCM, EPM, and SCM Fusion Clouds, are expected to help healthcare enterprises unify operations and improve patient care.

Other updates including distributed cloud offerings

Oracle also continued to expand its distributed cloud offerings, including Oracle Database@Azure and MySQL HeatWave Lakehouse on AWS.

As part of Database@Azure, the company is collocating its Oracle database hardware (including Oracle Exadata) and software in Microsoft Azure data centers, giving customers direct access to Oracle database services running on Oracle Cloud Infrastructure (OCI) via Azure.

Oracle Alloy, which serves as a cloud infrastructure platform for service providers, integrators, ISVs, and others who want to roll out their own cloud services to customers, has also been made generally available.

Next read this:

Posted Under: Database
Oracle’s MySQL HeatWave gets Vector Store, generative AI features

Posted by on 20 September, 2023

This post was originally published on this site

Oracle is adding a Vector Store and new generative AI features to its data analytics cloud service MySQL HeatWave, the company said at its annual CloudWorld conference.

MySQL HeatWave combines OLAP (online analytical processing), OLTP (online transaction processing), machine learning, and AI-driven automation in a single MySQL database.

The generative AI features added to the data analytics cloud service include a large language model-driven interface that allows enterprise users to interact with different aspects of the service — including searching for different files — in natural language.

The new Vector Store, which is also in private preview, can ingest documents in a variety of formats and store them as embeddings generated via an encoder model in order to process queries faster, the company said.

“For a given user query, the Vector Store identifies the most similar documents by performing a similarity search over the stored embeddings and the embedded query,” an Oracle spokesperson said. These documents can be later used to augment the prompt given to the LLM-driven interface so that it returns a more contextual answer.

AutoML support for MySQL HeatWave Lakehouse

Oracle’s MySQL HeatWave Lakehouse, which was released last year in October, has been updated to support AutoML.

HeatWave’s AutoML, which is a machine learning component or feature within the service, supports training, inference, and explanations on data in object storage in addition to data in the MySQL database, the company said.

Other updates to AutoML include support for text columns, an enhanced recommender system, and a training progress monitor.

Support for text columns, according to the company, will now allow enterprises to run various machine learning tasks — including anomaly detection, forecasting, classification, and regression — on data stored in these columns.

In March, Oracle added several new machine-learning features to MySQL HeatWave including AutoML and MySQL Autopilot.

Oracle’s recommender system — a recommendation engine within AutoML — has also been updated to support wider feedback, including implicit feedback, such as past purchases and browsing history, and explicit feedback, such as ratings and likes, in order to generate more accurate personalized recommendations.

A separate component, dubbed the Training Progress Monitor, has also been added to AutoML in order to allow enterprises to monitor the progress of their models being trained with HeatWave.

MySQL Autopilot to support automated indexing

Oracle has also updated its MySQL Autopilot component within HeatWave to support automatic indexing.

The new feature, which is currently in limited availability, is targeted at helping enterprises to eliminate the need to create optimal indexes for their OLTP workloads and maintain them as workloads evolve.

MySQL Autopilot automatically determines the indexes customers should create or drop from their tables to optimize their OLTP throughput, using machine learning to make a prediction based on individual application workloads,” the company said in a statement.

Another feature, dubbed auto compression, has also been added to Autopilot. Auto compression helps enterprises determine the optimal compression algorithm for each column, which improves load, and query performance and reduces cost.

The other updates in Autopilot include adaptive query execution and auto load and unload.

Adaptive query execution, as the name suggests, helps enterprises optimize the execution plan of a query in order to improve performance by using information obtained from the partial execution of the query to adjust data structures and system resources.

Separately, auto load and unload improve performance by automatically loading columns that are in use to HeatWave and unloading columns that are never in use.

“This feature automatically unloads tables that were never or rarely queried. This helps free up memory and reduce costs for customers, without having to manually perform this task,” the company said.

Other MySQL HeatWave enhancements

Oracle is also adding support for JavaScript to MySQL HeatWave. This ability, which is currently in limited availability, will allow developers to write stored procedures and functions in JavaScript and later execute them in the data analytics cloud service.

Other updates include JSON acceleration, new analytic operators for migrating more workloads into HeatWave, and a bulk ingest feature into MySQL HeatWave.

The bulk ingest feature adds support for parallel building of index sub-tress while loading data from CSV files. This provides a performance increase in data ingestion, thereby allowing newly loaded data to be queried sooner, the company said.

Next read this:

Posted Under: Database
Oracle’s Database 23c gets vector search to underpin generative AI use cases

Posted by on 20 September, 2023

This post was originally published on this site

Oracle is planning to add vector search capabilities to its database offering, dubbed Database 23c, the company announced at its ongoing annual CloudWorld conference.

These capabilities, dubbed AI Vector Search, include a new vector data type, vector indexes, and vector search SQL operators that enable the Oracle Database to store the semantic content of documents, images, and other unstructured data as vectors, and use these to run fast similarity queries, the company said.

AI Vector Search in Database 23c also supports Retrieval Augmented Generation (RAG), which is a generative AI technique that combines large language models (LLMs) and private business data to deliver responses to natural language questions, it added.

The addition of vector search capabilities to the Oracle database offering will allow enterprises to add an LLM-based natural language interface inside applications built on the Oracle Database and Autonomous Database.

The natural language-based interface, according to the company, enables users of the applications to ask questions about their data without having to write any code.

“Autonomous Database takes an open, flexible API approach to integrating LLMs so that developers can select the best LLM from Oracle or third parties to generate SQL queries to respond to these questions,” Oracle said in a statement.

The company said it will also add generative AI capabilities to Oracle Database tools such as APEX and SQL Developer. These enhancements will allow developers to use natural language to generate applications or generate code for SQL queries.

Additionally, a new integrated workflow and process automation capability has been added to APEX that will allow developers to add different functions to applications, such as invoking actions, triggering approvals, and sending emails.

However, Oracle did not announce the pricing and availability details for any of the new capabilities.

Updates to other database offerings

Other updates to Oracle’s database offerings include the general availability of Database 23c, the next generation of Exadata Exascale, and updates to its Autonomous Database service and GoldenGate 23c.

The next generation of Oracle’s Exadata Exascale, which is system software for databases, according to the company, lowers the cost of running Exadata for cloud databases for developers using smaller configurations.

Updates to Oracle’s Autonomous Database include Oracle’s Globally Distributed Autonomous Database and Elastic Resource Pools.

While the fully managed Globally Distributed Autonomous Database service helps enterprises simplify the development and deployment of shared or distributed application architectures for mission-critical applications, Elastic Resource Pools is designed to enable enterprises to consolidate database instances without any downtime to reduce cost.

For enterprises using Oracle hardware to run databases, the company has introduced Oracle Database Appliance X10.

“The latest release of this database-optimized engineered system provides enhanced end-to-end automation, and up to 50% more performance than the previous generation,” the company said in a statement.

Next read this:

Posted Under: Database
DataStax’s new JSON API targets JavaScript developers

Posted by on 19 September, 2023

This post was originally published on this site

DataStax on Tuesday said that it was releasing a new JSON API in order to help JavaScript developers leverage its  serverlessNoSQL Astra DB as a vector database for their large language model (LLMs), AI assistant, and real-time generative AI projects.

Vector search, or vectorization, especially in the wake of generative AI proliferation, is seen as a key capability by database vendors as it can reduce the time required to train AI models by cutting down the need to structure data — a practice prevalent with current search technologies. In contrast, vector searches can read the required or necessary property attribute of a data point that is being queried.  

The addition of the new JSON API will eliminate the need for developers trained in JavaScript to have a deep understanding of Cassandra Query Language (CQL) in order to work with Astra DB as the database is based on Apache Cassandra, the company said.

This means that these developers can continue to write code in the language that they are familiar with, thereby reducing the time required to develop AI-based applications which are in demand presently, it added.

Further, the new API, which can be accessed via DataStax’s open source API gateway, dubbed Stargate, will also provide compatibility with Mongoose — one of the most popular open source object data modeling library for MongoDB.

In October last year, DataStax launched the second version of its open-source data API gateway, dubbed Stargate V2, just months after making its managed Astra Streaming service generally available.

In June this year, the company partnered with Google to bring vector search to Astra DB.

Next read this:

Posted Under: Database
A deep dive into caching in Presto

Posted by on 19 September, 2023

This post was originally published on this site

Presto is a popular, open source, distributed SQL engine that enables organizations to run interactive analytic queries on multiple data sources at a large scale. Caching is a typical optimization technique for improving Presto query performance. It provides significant performance and efficiency improvements for Presto platforms.

Caching avoids expensive disk or network trips to refetch data by storing frequently accessed data in memory or on fast local storage, speeding up overall query execution. In this article, we provide a deep dive into Presto’s caching mechanisms and how you can use them to boost query speeds and reduce costs.

Benefits of caching

Caching provides three key advantages. By implementing caching in Presto, you can:

  1. Boost query performance. Caching frequently accessed data allows Presto to retrieve results from faster and closer caches rather than scanning slower storage. For repetitive analytical queries, this can improve query speeds by orders of magnitude, reducing overall latency. By accelerating query execution, caching enables interactive querying and faster time-to-insight.
  2. Reduce infrastructure costs. Caching reduces the volume of data read from remote storage systems like S3, resulting in lower egress charges and charges for storage API requests. For data stored in the cloud, caching minimizes repetitive retrieval of data over the network. This provides substantial cost savings, especially for large datasets.
  3. Minimize network overhead. By reducing unnecessary data transfer between Presto components and remote storage, caching alleviates network congestion. Local caching prevents bottlenecking of network links between distributed Presto workers. It also reduces load and bandwidth usage on connections to external data sources.

Overall, caching can boost performance and efficiency of Presto queries, providing significant value and ROI for Presto-based analytics platforms.

Different types of caching in Presto

There are two types of caches in Presto, the built-in cache and third-party caches. The built-in cache includes the metastore cache, file list cache, and Alluxio SDK cache. It uses the memory and SSD resources of the Presto cluster, running within the same process as Presto for optimal performance.

The main benefits of built-in caches are very low latency and no network overhead because data is cached locally within the Presto cluster. However, built-in cache capacity is constrained by worker node resources.

Third-party caches, such as the Alluxio distributed cache, are independently deployable and offer better scalability and increased cache capacity. They are particularly advantageous for large-scale analytics workloads, cross-region/cloud deployments, and reducing API and egress costs for cloud storage.

presto caching 01 Alluxio

The diagram above and table below summarizes the different cache types, their corresponding resource types, locations.

Type of cache

Cache location

Resource type

Metastore cache

Presto coordinator


List file cache

Presto coordinator


Alluxio SDK cache

Presto workers


Alluxio distributed cache

Alluxio workers


None of Presto’s caches are enabled by default. You will need to modify Presto’s configuration to activate them. We will explain the different caching types in more detail and how to enable them via configuration properties in the following sections.

Metastore cache

Presto’s metastore cache stores Hive metastore query results in memory for faster access. This reduces planning time and metastore requests.

The metastore cache is highly beneficial when the Hive metastore is overloaded. For large partitioned tables, the cache stores partition metadata locally, enabling faster access and fewer repeated queries. This decreases the overall load on the Hive metastore.

To enable metastore cache, use the following settings:


Note that, if tables are frequently updated, you should configure a shorter TTL or refresh interval for the metastore versioned cache. A shorter cache refresh interval ensures only current metadata is stored, reducing the risk of outdated metadata in query execution. This prevents Presto from using stale data.

List file status cache

The list file cache stores file paths and attributes to avoid repeated retrievals from the namenode or object store.

The list file cache substantially improves query latency when the HDFS namenode is overloaded or object stores have poor file listing performance. List file calls can bottleneck HDFS, overwhelming the name node, and increase costs for S3 storage. When the list file status cache is enabled, the Presto coordinator caches file lists in memory for faster access to frequently used data, reducing lengthy remote listFile calls.

To configure list file status caching, use the following settings:


Note that the list file status cache can be applied only to sealed directories, as Presto skips caching open partitions to ensure data freshness.

Alluxio SDK cache (native)

The Alluxio SDK cache is a Presto built-in cache that reduces table scan latency. Because Presto is a storage-agnostic engine, its performance is often bottlenecked by storage. Caching data locally on Presto worker SSDs enables fast query access and execution. By minimizing repeated network requests, the Alluxio cache also reduces cloud egress fees and storage API costs for remote data.

The Alluxio SDK cache is particularly beneficial for querying remote data like cross-region or hybrid cloud object stores. This significantly decreases query latency and associated cloud storage egress costs and API costs.

Enable the Alluxio SDK cache with the settings below:


To achieve the best cache hit rate, change the node selection strategy to soft affinity:

presto caching 02Alluxio

The diagram above shows the soft-affinity node selection architecture. Soft-affinity scheduling attempts to send requests to workers based on file paths, maximizing cache hits by locating data in worker caches. Soft affinity is “soft” because it is not a strict rule—if the preferred worker is busy, the split is sent to another available worker rather than waiting.

If you encounter errors such as “Unsupported Under FileSystem,” download the latest Alluxio client JAR from the Maven repository and place it in the {$presto_root_path}/plugin/hive-hadoop2/ directory.

You can view the full documentation here.

Alluxio distributed cache (third-party)

If Presto memory or storage is insufficient for large datasets, using a third-party caching solution provides expansive caching for frequent data access. A third-party cache can deliver several optimizations for Presto:

  • Improve performance by reducing I/O latency
  • Accelerate queries on remote cross-datacenter or cloud data storage
  • Provide a shared cache between Presto workers, clusters, and other engines like Apache Spark
  • Enables resilient caching for cost savings on spot instances

The Alluxio distributed cache is one example of a third-party cache. As you can see in the diagram below, the Alluxio distributed cache is deployed between Presto and storage like S3. Alluxio uses a master-worker architecture where the master manages metadata and workers manage cached data on local storage (memory, SSD, HDD). On a cache hit, the Alluxio worker returns data to the Presto worker. Otherwise, the Alluxio worker retrieves data from persistent storage and caches data for future use. Presto workers process the cached data and the coordinator returns results to the user.

presto caching 03Alluxio

Here are the steps to deploy Alluxio distributed caching with Presto.

1. Distribute the Alluxio client JAR to all Presto servers

In order for Presto to be able to communicate with the Alluxio servers, the Alluxio client jar must be in the classpath of Presto servers. Put the Alluxio client JAR /<PATH_TO_ALLUXIO>/client/alluxio-2.9.3-client.jar into the directory ${PRESTO_HOME}/plugin/hive-hadoop2/ on all Presto servers. Restart the Presto workers and coordinator using the command below:

$ ${PRESTO_HOME}/bin/launcher restart

2. Add Alluxio Configurations to Presto’s HDFS configuration files

You can add Alluxio’s properties to the HDFS configuration files such as core-site.xml and hdfs-site.xml, and then use the Presto property hive.config.resources in the file ${PRESTO_HOME}/etc/catalog/ to point to the locations of HDFS configuration files on every Presto worker.


Then, add the property to the HDFS core-site.xml configuration, which is linked by hive.config.resources in Presto’s property.


Based on the configuration above, Presto is able to locate the Alluxio cluster and forward the data access to it.

To learn more about Alluxio distributed cache for Presto, follow this documentation.

Choosing the right cache for your use case

Caching is a powerful way to improve Presto query performance. In this article, we have introduced different caching mechanisms in Presto, including the metastore cache, the list file status cache, the Alluxio SDK cache, and the Alluxio distributed cache. As summarized in the table below, you can use these caches to accelerate data access based on your use case.

Type of cache

When to use

Metastore cache

Slow planning time
Slow Hive metastore
Large tables with hundreds of partitions

List file status cache

Overloaded HDFS namenode
Overloaded object store like S3

Alluxio SDK cache

Slow or unstable external storage

Alluxio distributed cache

Cross-region, multicloud, hybrid cloud
Data sharing with other compute engines

The Presto and Alluxio open-source communities work continuously to improve the existing caching features and to develop new capabilities to enhance query speeds, optimize efficiency, and improve the system’s scalability and reliability.


Beinan Wang is senior staff engineer at Alluxio. He has 15 years of experience in performance optimization and large-scale data processing. He is a PrestoDB committer and contributes to the Trino project. He previously led Twitter’s Presto team. Beinen earned his Ph.D. in computer engineering from Syracuse University, specializing in distributed systems.

Hope Wang is developer advocate at Alluxio. She has a decade of experience in data, AI, and cloud. An open source contributor to Presto, Trino, and Alluxio, she also holds AWS Certified Solutions Architect – Professional status. Hope earned a BS in computer science, a BA in economics, and an MEng in software engineering from Peking University and an MBA from USC.

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to

Next read this:

Posted Under: Database
Kinetica offers its own LLM for SQL queries, citing security, privacy concerns

Posted by on 18 September, 2023

This post was originally published on this site

Citing privacy and security concerns over public large language models, Kinetica is adding a self-developed LLM for generating SQL queries from natural language prompts to its relational database for online analytical processing (OLAP) and real-time analytics.

The company, which derives more than half of its revenue from US defense organizations such as NORAD and the Air Force, claims that the native LLM is more secure, tailored to the database management system syntax, and is contained within the customer’s network perimeter.

With the release of its LLM, Kinetica joins the ranks of all the major LLM or generative AI services providers — including IBM, AWS, Oracle, Microsoft, Google, and Salesforce — that claim that they keep enterprise data to within their respective containers or servers. These providers also claim that customer data is not used to train any large language model.

In May, Kinetica, which offers its database in multiple flavors including hosted, SaaS and on-premises, had said that it would integrate OpenAI’s ChatGPT to let developers use natural language processing to do SQL queries.  

Further, the company said that it was working to add more LLMs to its database offerings, including Nvidia’s NeMo model.

The new LLM from Kinetica also gives enterprise users the capability to handle other tasks such as querying time-series graph and spatial queries for better decision making, the company said in a statement.

The native LLM is immediately available to customers in a containerized, secure environment either on-premises or in the cloud without any additional cost, it added.

Next read this:

Posted Under: Database
4 key new features in PostgreSQL 16

Posted by on 14 September, 2023

This post was originally published on this site

Today, the PostgreSQL Global Development Group shared the release of PostgreSQL 16. With this latest update, Postgres sets new standards for database management, data replication, system monitoring, and performance optimization, marking a significant milestone for the community, developers, and EDB as the leading contributor to PostgreSQL code.

With PostgreSQL 16 comes a plethora of new features and enhancements. Let’s take a look at a few of the highlights.

Privilege admin

One of the standout changes in PostgreSQL 16 is the overhaul of privilege administration. Previous versions often required a superuser account for many administrative tasks, which could be impractical in larger organizations with multiple administrators. PostgreSQL 16 addresses this issue by allowing users to grant privileges in roles only if they possess the ADMIN OPTION for those roles. This shift empowers administrators to define more specific roles and assign privileges accordingly, streamlining the management of permissions. This change not only enhances security but also simplifies the overall user management experience.

Logical replication enhancements

Logical replication has been a flexible solution for data replication and distribution since it was first included with PostgreSQL 10 nearly six years ago, enabling various use cases. There have been enhancements to logical replication in every Postgres release since, and Postgres 16 is no different. This release not only includes necessary under-the-hood improvements for performance and reliability but also the enablement of new and more complex architectures.

With Postgres 16, logical replication from physical replication standbys is now supported. Along with helping reduce the load on the primary, which receives all the writes in the cluster, easier geo-distribution architectures are now possible. The primary might have a replica in another region, which can send data to a third system in that region rather than replicating the data twice from one region to another. The new pg_log_standby_snapshot() function makes this possible.

Other logical replication enhancements include initial table synchronization in binary format, replication without a primary key, and improved security by requiring subscription owners to have SET ROLE permissions on all tables in the replication set or be a superuser.

Performance boosts

PostgreSQL 16 doesn’t hold back when it comes to performance improvements. Enhanced query execution capabilities allow for parallel execution of FULL and RIGHT JOINs, as well as the string_agg and array_agg aggregate functions. SELECT DISTINCT queries benefit from incremental sorts, resulting in better performance. The concurrent bulk loading of data using COPY has also seen substantial performance enhancements, with reported improvements of up to 300%.

This release also introduces features like caching RANGE and LIST partition lookups, which help with bulk data loading in partitioned tables and better control of shared buffer usage by VACUUM and ANALYZE, ensuring your database runs more efficiently than ever.

Comprehensive monitoring features

Monitoring PostgreSQL databases has never been more detailed or comprehensive. PostgreSQL 16 introduces the pg_stat_io view, allowing for better insight into the I/O activity of your Postgres system. System-wide IO statistics are now only a query away, allowing you to see read, write, and extend (back-end resizing of data files) activity by different back-end types, such as VACUUM and regular client back ends.

PostgreSQL 16 records statistics on the last sequential and index scans on tables, adds speculative lock information to the pg_locks view, and makes several improvements to wait events that make monitoring of PostgreSQL more comprehensive than ever.

What makes PostgreSQL 16 truly exceptional is its potential to impact not just PostgreSQL users, but the entire industry. EDB’s commitment to the community and customers has culminated in a robust, secure, and user-centric database system that promises innovation and productivity across sectors. That’s why EDB builds enterprise-ready capabilities on top of Postgres in EDB Postgres Advanced Server, with features such as Privilege Analysis and new options for Transparent Data Encryption coming out this November.

Additionally, PostgreSQL 16 debuts on EDB BigAnimal next month. This cloud-ready, enterprise-grade database-as-a-service platform is available to organizations worldwide, enabling them to harness the full power of PostgreSQL 16 in their preferred public cloud environments.

Adam Wright is the product manager of core database, extensions, and backup/restore at EDB.

New Tech Forum provides a venue for technology leaders—including vendors and other outside contributors—to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to

Next read this:

Posted Under: Database
Page 2 of 1112345...10...Last »

Social Media

Bulk Deals

Subscribe for exclusive Deals

Recent Post



Subscribe for exclusive Deals

Copyright 2015 - InnovatePC - All Rights Reserved

Site Design By Digital web avenue