Like every other programming environment, you need a place to store your data when coding in the browser with JavaScript. Beyond simple JavaScript variables, there are a variety of options ranging in sophistication, from using localStorage to cookies to IndexedDB and the service worker cache API. This article is a quick survey of the common mechanisms for storing data in your JavaScript programs.
JavaScript variables
You are probably already familiar with JavaScript’s set of highly flexible variable types. We don’t need to review them here; they are very powerful and capable of modeling any kind of data from the simplest numbers to intricate cyclical graphs and collections.
The downside of using variables to store data is that they are confined to the life of the running program. When the program exits, the variables are destroyed. Of course, they may be destroyed before the program ends, but the longest-lived global variable will vanish with the program. In the case of the web browser and its JavaScript programs, even a single click of the refresh button annihilates the program state. This fact drives the need for data persistence; that is, data that outlives the life of the program itself.
An additional complication with browser JavaScript is that it’s a sandboxed environment. It doesn’t have direct access to the operating system because it isn’t installed. A JavaScript program relies on the agency of the browser APIs it runs within.
Saving data on the server
The other end of the spectrum from using built-in variables to store JavaScript data objects is sending the data off to a server. You can do this readily with a fetch() POST request. Provided everything works out on the network and the back-end API, you can trust that the data will be stored and made available in the future with another GET request.
So far, we’re choosing between the transience of variables and the permanence of server-side persistence. Each approach has a particular profile in terms of longevity and simplicity. But a few other options are worth exploring.
Web storage API
There are two types of built-in “web storage” in modern browsers: localStorage and sessionStorage. These give you convenient access to longer-lived data. They both give you a key-value and each has its own lifecycle that governs how data is handled:
localStorage saves a key-value pair that survives across page loads on the same domain.
sessionStorage operates similarly to localStorage but the data only lasts as long as the page session.
In both cases, values are coerced to a string, meaning that a number will become a string version of itself and an object will become “[object Object].” That’s obviously not what you want, but if you want to save an object, you can always use JSON.stringify() and JSON.parse().
Both localStorage and sessionStorage use getItem and setItem to set and retrieve values:
You can most clearly see the difference between the two by setting a value on them and then closing the browser tab, then reopening a tab on the same domain and checking for your value. Values saved using localStorage will still exist, whereas sessionStorage will be null. You can use the devtools console to run this experiment:
localStorage.setItem("foo",”bar”);
sessionStorage.setItem("foo","bar");
// close the tab, reopen it
localStorage.getItem('bar2'); // returns “bar”
sessionStorage.getItem("foo") // returns null
Cookies
Whereas localStorage and sessionStorage are tied to the page and domain, cookies give you a longer-lived option tied to the browser itself. They also use key-value pairs. Cookies have been around for a long time and are used for a wide range of cases, including ones that are not always welcome. Cookies are useful for tracking values across domains and sessions. They have specific expiration times, but the user can choose to delete them anytime by clearing their browser history.
Cookies are attached to requests and responses with the server, and can be modified (with restrictions governed by rules) by both the client and the server. Handy libraries like JavaScript Cookie simplify dealing with cookies.
Cookies are a bit funky when used directly, which is a legacy of their ancient origins. They are set for the domain on the document.cookie property, in a format that includes the value, the expiration time (in RFC 5322 format), and the path. If no expiration is set, the cookie will vanish after the browser is closed. The path sets what path on the domain is valid for the cookie.
Here’s an example of setting a cookie value:
document.cookie = "foo=bar; expires=Thu, 18 Dec 2024 12:00:00 UTC; path=/";
And to recover the value:
function getCookie(cname) {
const name = cname + "=";
const decodedCookie = decodeURIComponent(document.cookie);
const ca = decodedCookie.split(';');
for (let i = 0; i < ca.length; i++) {
let c = ca[i];
while (c.charAt(0) === ' ') {
c = c.substring(1);
}
if (c.indexOf(name) === 0) {
return c.substring(name.length, c.length);
}
}
return "";
}
const cookieValue = getCookie("foo");
console.log("Cookie value for 'foo':", cookieValue);
In the above, we use decodeURIComponent to unpack the cookie and then break it along its separator character, the semicolon (;), to access its component parts. To get the value we match on the name of the cookie plus the equals sign.
IndexedDB is the most elaborate and capable in-browser data store. It’s also the most complicated. IndexedDB uses asynchronous calls to manage operations. That’s good because it lets you avoid blocking the thread, but it also makes for a somewhat clunky developer experience.
IndexedDB is really a full-blown object-oriented database. It can handle large amounts of data, modeled essentially like JSON. It supports sophisticated querying, sorting, and filtering. It’s also available in service workers as a reliable persistence mechanism between thread restarts and between the main and workers threads.
When you create an object store in IndexedDB, it is associated with the domain and lasts until the user deletes it. It can be used as an offline datastore to handle offline functionality in progressive web apps, in the style of Google Docs.
To get a flavor of using IndexedDB, here’s how you might create a new store:
let db = null; // A handle for the DB instance
llet request = indexedDB.open("MyDB", 1); // Try to open the “MyDB” instance (async operation)
request.onupgradeneeded = function(event) { // onupgradeneeded is the event indicated the MyDB is either new or the schema has changed
db = event.target.result; // set the DB handle to the result of the onupgradeneeded event
if (!db.objectStoreNames.contains("myObjectStore")) { // Check for the existence of myObjectStore. If it doesn’t exist, create it in the next step
let tasksObjectStore = db.createObjectStore("myObjectStore", { autoIncrement: true }); // create myObjectStore
}
};
The call to request.onsuccess = function(event) { db = event.target.result; }; // onsuccess fires when the database is successfully opened. This will fire without onupgradeneeded firing if the DB and Object store already exist. In this case, we save the db reference:
request.onerror = function(event) { console.log("Error in db: " + event); }; // If an error occurs, onerror will fire
The above IndexedDB code is simple—it just opens or creates a database and object store—but the code gives you a sense of IndexedDB‘s asynchronous nature.
Service worker cache API
Service workers include a specialized data storage mechanism called cache. Cache makes it easy to intercept requests, save responses, and modify them if necessary. It’s primarily designed to cache responses (as the name implies) for offline use or to optimize response times. This is something like a customizable proxy cache in the browser that works transparently from the viewpoint of the main thread.
Here’s a look at caching a response using a cache-first strategy, wherein you try to get the response from the cache first, then fallback to the network (saving the response to the cache):
self.addEventListener('fetch', (event) => {
const request = event.request;
const url = new URL(request.url);
// Try serving assets from cache first
event.respondWith(
caches.match(request)
.then((cachedResponse) => {
// If found in cache, return the cached response
if (cachedResponse) {
return cachedResponse;
}
// If not in cache, fetch from network
return fetch(request)
.then((response) => {
// Clone the response for potential caching
const responseClone = response.clone();
// Cache the new response for future requests
caches.open('my-cache')
.then((cache) => {
cache.put(request, responseClone);
});
return response;
});
})
);
});
This gives you a highly customizable approach because you have full access to the request and response objects.
Conclusion
We’ve looked at the commonly used options for persisting data in the browser of varying profiles. When deciding which one to use, a useful algorithm is: What is the simplest option that meets my needs? Another concern is security, especially with cookies.
Other interesting possibilities are emerging with using WebAssembly for persistent storage. Wasm’s ability to run natively on the device could give performance boosts. We’ll look at using Wasm for data persistence another day.
EDB Postgres AI, which was internally named Project Beacon during its development, started its life as a data lakehouse project with support for Delta Live Tables and later evolved into a product that combines EDB’s PostgreSQL software and other components such as data lakehouse analytics into a singular unified offering.
PostgreSQL, which is an object-oriented relational database, has been gaining popularity because it is an open-source database with many potential providers and can be adapted to multiple workloads, said Matt Aslett, director at ISG’s Ventana Research.
“As a general-purpose database, PostgreSQL is suitable for both transactional and analytic applications,” Aslett explained.
The huge ecosystem of PostgreSQL-based databases that leverage the core technology and skills base makes PostgreSQL impossible to ignore, positioning it as a default standard enterprise-grade open-source database, experts said.
According to data from database knowledge base DB-Engines, PostgreSQL has been steadily rising in popularity and is currently the fourth most popular RDBMS (relational database management system) and fourth most popular database product overall in their rankings.
The components of of EDB Postgres AI, which the company describes as an “intelligent data platform,” include a central management console with AI assistance, EDP Postgres databases, data lakehouse analytics, and AI/ML including vector database support.
The console, according to the company, provides a single pane to all EDB Postgres AI operations and helps manage the database landscape of an enterprise, in turn providing better observability. The console comes with an AI agent that can help enterprises manage on-premises databases.
EDB Postgres AI supports most databases that EDB offers including EDB Postgres, EDB Postgres Advanced Server, EDB Postgres Extended Server, and their respective distributed high availability versions. EDB Postgres Advanced Server also provides Oracle Database compatibility, the company said.
The lakehouse analytics module, according to EDB, brings structured and unstructured data together with the help of Nodes to be analyzed. Nodes support multiple formats, the company said, adding that it has built a custom store to support multiple data formats.
The AI/ML module includes vector support, which effectively gives the platform its capability to build AI-powered applications.
Additionally, Postgres AI comes with support for extensions such as Postgres Enterprise Manager, Barman, Query Advisor, and migration tools, such as the Migration Toolkit and the Migration Portal.
The Migration Portal, according to the company, is among the first EDB tools to include embedded AI via an AI copilot that can assist users in developing migration strategies.
The combination of these components or modules result in Postgres AI’s key capabilities such as rapid analytics, observability, vector support, high availability, and legacy modernization.
Explaining rapid analytics as a capability, EDB said that Postgres AI allows enterprises to spin up analytics clusters on demand.
“With EDB Postgres Lakehouse capabilities, operational data can be stored in a columnar format, optimizing it for fast analytics,” the company said in a statement.
EDB added that its acquisition of Splitgraph, a startup that provides a PostgreSQL-compatible serverless SQLAPI for building data-driven applications from multiple data sources, last year played a foundational role in building out the analytics capability.
The release of EDB Postgres AI saw the company partner with the likes of Red Hat, Nutanix, and SADA.
EDB’s collaboration with Red Hat will enable enterprises to build AI models on Red Hat OpenShift AI and deliver enterprise-grade, day-two operations with EDB Postgres AI, the company said.
EDB
EDB Postgres AI availability and pricing
EDB Postgres AI, according to Jozef de Vries, the chief engineering officer at EDB, is available as a managed service on AWS, GCP, and Azure.
“The analytics functionality is initially available only on AWS, with the other public clouds to follow soon,” Vries said, adding that Postgres AI can also be self-managed on a public cloud and private cloud environment of the customer’s choice.
EDB prices its EDB Postgres offerings on a per vCPU-hour basis. The company also provides a free tier across all of its database offerings.
The EDB Postgres offering costs $0.0856 per vCPU-hour, the company’s subscription listing showed. Other options, such as EDB Postgres Extended Server, EDB Postgres Advanced Server, and their distributed high availability versions cost $0.1655 per vCPU-hour, $0.2568 per vCPU-hour, $0.3424 per vCPU-hour, and $0.2511 per vCPU-hour respectively.
Why is EDB launching Postgres AI?
EDB Postgres AI, according to Aslett of Ventana Research, is being positioned to help enterprises bring AI capabilities to a variety of workloads regardless of deployment location.
“With EDB Postgres AI, EDB is addressing the requirement for data storage and processing both on-premises and in the cloud. This is important for AI-infused applications, which require high-performance AI inference at the point of interaction,” the research director explained.
Omdia’s chief analyst Bradley Shimmin sees the release of EDB Postgres AI as the repeat of the market branding mania from 1980s.
“It does seem like the 1980s with its ‘Turbo’ market branding mania, as we’re seeing a sudden and very pervasive influx of AI-branded products entering the market,” Shimmin said. Shimmin cited recent examples such as Red Hat Enterprise Linux AI and Oracle Database 23ai.
Shimmin said that he sees the EDB Postgres AI release as a mix of marketing hype and maturation of Postgres offerings.
“The Postgres database was certainly capable of supporting AI workloads before this release, plying vector data as a means of steering large language models away from confabulation and toward fact. What we are seeing from vendors like EDB with these AI branded releases is the vertical integration of functionality geared toward streamlining, simplifying, and accelerating AI-infused use cases,” the analyst explained.
The release of EDB Postgres AI coincides with EDB’s strategic move to a new corporate identity, EDB said.
There appears to be many questions and few answers about MariaDB plc’s long-term strategy following an announcement that its shareholders have accepted an offer by California-based investment firm K1 Investment Management.
News that the company that provides database and SaaS services around the open-source database MariaDB had been acquired came on Monday, when it was announced that a trio of companies—K1; Meridian Bidco LLC, a K1 affiliate; and K5 Capital Advisors—“now have irrevocable shareholder support in respect of 68.51% of MariaDB shares.”
The company has had a litany of financial issues over the past 12 months, but when it released financial results for the quarter ended March 31 last week there was one bright spot: Its net loss had shrunk to $3.5 million, compared to a net Loss of $11.9 million a year earlier.
“We have demonstrated our ability to quickly turn our financial story around and are optimistic about the future performance of the business,” Paul O’Brien, CEO of MariaDB plc said in a statement accompanying the financial results.
As reported in InfoWorld in February, after going public in December 2022, the company saw its market capitalization plummet from $445 million to around $10 million by the end of 2023.
Carl Olofson, research vice president and database analyst with IDC, said that the key to determining what happens next is why the acquisition happened in the first place.
While executives at K1 and MariaDB plc have yet to comment on their future plans, Olofson said that “when you see something like this, there is one of two motivations. One is that you want to dismantle the company, and make a profit from the assets, which is not going to be the case here, because they do not really have assets.
“The other option is to really believe that with proper management, the right approach, the company can grow far beyond where it’s at now—make fabulous profits, sell it off and everybody walks away happy.”
In the open source database space, he said, there is a big difference when it comes to intellectual property (IP) between MariaDB and MySQL, the open-source database of which MariaDB is a fork. In the case of MySQL, the IP is owned by Oracle, but for MariaDB “it is owned by the MariaDB community, which is not part of the company. There is a clear distinction between MariaDB, the company, and MariaDB, the community.”
Olofson added that regardless of what happens to the corporate entity, the community will continue.
The open source project was created by Michael “Monty” Widenius, who was also a creator of MySQL. He set up the company Monty Program Ab which later became MariaDB, the company, and also the MariaDB Foundation, which is the custodian of the project’s open-source code.
Olofson spoke with Widenius briefly last year at a MariaDB user conference and described him as an “open source purist” who wants to “just put technology out there and let people do what they will with it.” But while that attitude might be great for users, it’s not good for MariaDB, the company: “That does not really help it in trying to survive commercially and meet payroll,” Olofson said.
As for the commercial side, while Olofson has no idea what K1’s corporate strategy might be moving forward, he said one option is to “go down the well-worn path of other open source software companies that are trying to make a living from the technology by what’s called an open core approach.”
That was already MariaDB’s strategy—it offers paid add-ons such as MaxScale, ColumnStore, or Galera Cluster, as well as consulting, migration and managed cloud services—“but they basically just ran out of money,” said Olofson.
“They were adding some really exciting and innovative features ,” he said, “and then they pulled back from some of that. I interpreted that as meaning they did not have the money to continue to support development.”
InfoWorld reached out to both MariaDB plc and K1 for comment, but at press time had not heard from either organization.
Lift the hood on most business applications, and you’ll find they have some way to store and use structured data. Whether it’s a client-side app, an app with a web front end, or an edge-device app, chances are a business application needs a database. In many cases, an embedded database will do. Embedded databases are lightweight, compact, and portable—and for some applications, they are a better choice than a traditional server.
SQLite is an embeddable open source database, written in C and queryable with conventional SQL. SQLite is designed to be fast, portable, and reliable, whether you’re storing only kilobytes of data or multi-gigabyte blobs. We’ll take a look at SQLite, including where and when to use it and how it compares to alternatives such as MySQL, MariaDB, and other popular embedded databases.
What is SQLite used for?
The most common and obvious use case for SQLite is serving as a conventional, table-oriented relational database. SQLite supports transactions and atomic behaviors, so a program crash or even a power outage won’t leave you with a corrupted database. SQLite also has other features found in higher-end databases, such as full-text indexing, and support for large databases—up to 281 terabytes with row sizes up to 1GB.
SQLite also provides a fast and powerful way to store configuration data for a program. Instead of parsing a file format like JSON or YAML, a developer can use SQLite as an interface to those files—often far faster than operating on them manually. SQLite can work with in-memory data or external files (e.g., CSV files) as if they were native database tables, providing a handy way to query that data. It also natively supports JSON data, so data can be stored as JSON or queried in-place.
Advantages of SQLite
SQLite has many advantages, starting with its platform and language portability. Here are the main benefits of using SQLite:
It’s cross-platform: One of SQLite’s greatest advantages is that it can run nearly anywhere. SQLite has been ported to a wide variety of platforms: Windows, macOS, Linux, iOS, Android, and more. Windows users in particular can use precompiled binaries for regular Win32, UWP, WinRT, and .Net. Whatever your app’s deployment target is, odds are there’s an edition of SQLite available for it, or a way to port the C source code to that target.
It’s compatible with most programming languages: Applications that use SQLite don’t have to be written in any particular language, as long as there is some way to bind and work with external libraries written in C. SQLite’s binaries are self-contained, so they require no particular magic to deploy—you can simply drop them into the same directory as your application.
Yes, SQLite works with Python: Many languages have high-level bindings for SQLite as a library, and can use that in conjunction with other database access layers for the language. Python, for instance, bundles the SQLite library as a standard-issue element with the stock version of the Python interpreter. In addition, third parties have written a wide variety of ORMs and data layers that use SQLite, so you’re not stuck accessing SQLite via raw SQL strings (which is not only clumsy but also potentially dangerous).
Self-contained: Because SQLite is a single standalone binary, it’s easy to deploy with an app and then move with the app as needed. Each database created by SQLite also comprises a single file, which can be compacted or optimized using SQL commands.
Third-party extensions: Third-party binary extensions for SQLite add even more functionality. SQLCipher adds 256-bit AES encryption to SQLite database files. Another, sqlean, expands SQLite’s native functions to include many more not available by default, such as generating UUIDs or regular expression matching.
Extensive tooling: Many other third-party projects provide additional tooling for SQLite, such as the Visual Studio Code extension that allows browsing databases from within Visual Studio Code, or the LiteCLI interactive command-line for SQLite. A curated list of SQLite resources on GitHub includes many more options.
SQLite is open source: Finally, the source code for SQLite is public domain, so it can be reused in other programs with no practical restrictions.
SQLite vs. MySQL
SQLite is frequently compared to MySQL, the widely used open source database product that is a staple of today’s application stacks. As much as SQLite resembles MySQL, there are good reasons to favor one over the other, depending on the use case. The same is true for MariaDB, another popular database that is sometimes compared to SQLite.
Data types
SQLite has relatively few native data types—BLOB, NULL, INTEGER, REAL, and TEXT. Both MySQL and MariaDB, on the other hand, have dedicated data types for dates and times, various precisions of integers and floats, and much more.
If you’re storing relatively few data types, or you want to use your data layer to perform validation on the data, SQLite is useful. However, if you want your data layer to provide its own validation and normalization, go with MySQL or MariaDB.
Configuration and tuning
SQLite’s configuration and tuning options are minimal. Most of its internal or command-line flags deal with edge cases or backward compatibility. This fits with SQLite’s overall philosophy of simplicity: the default options are well-suited to most common use cases.
MySQL and MariaDB offer a veritable forest of database- and installation-specific configuration options—collations, indexing, performance tuning, storage engines, etc. The plethora of options is because these database products offer far more features. You may have to tweak them more, but it’s likely because you’re trying to do more in the first place.
Single-user vs. multi-user database
SQLite is best suited for applications with a single concurrent user, such as in desktop or mobile apps. MySQL and MariaDB are designed to handle multiple concurrent users. They can also provide clustered and scale-out solutions, whereas SQLite can’t.
Some projects add scaling features to SQLite, although not as a direct substitute for MySQL or MariaDB. Canonical has created its own variant of SQLite, dqlite, designed to scale out across a cluster. Data is kept consistent across nodes by way of a Raft algorithm, and deploying dqlite has only marginally more administrative overhead than SQLite.
SQLite vs. embedded databases
SQLite is far from the only embeddable database. Many others deliver similar features but emphasize different use cases or deployment models.
Apache Derby: An embeddable SQL engine, also repackaged by Oracle as Java DB. Since Apache Derby is written in Java and requires the JVM, it’s mainly designed for embedding in Java apps.
Firebird Embedded: The Firebird database, which runs cross-platform and sports many high-end features, is available as a library that can be embedded in a client application. Its feature set compares well to SQLite, but SQLite has a far larger user community and support base.
Realm: A high-performance relational database designed for mobile environments (mainly Android) that is also able to support desktop environments like Windows. Realm is object-based, however, and doesn’t use SQL queries—good if you’d rather not use SQL, but bad if SQL is familiar and comfortable. Realm is now a MongoDB project, and comes with the caveat that it is “not in itself an ‘end-user’ product with a publicly stable and supported API.”
VistaDB: An embedded database for the .Net runtime. VistaDB is available in versions specific to the various flavors and incarnations of .Net and with many enterprise features like full-database encryption. However, it’s a commercial product, not open source.
Berkeley DB: An Oracle project, nominally a key/value store, but one that uses SQLite in recent editions as a way to handle SQL queries. Berkeley DB’s underlying database engine has performance enhancements that SQLite can’t match, such as being able to handle multiple simultaneous write operations. Berkeley DB is dual-licensed, under either the GNU Affero GPL 3 or via a commercial license, depending on your use case.
Limitations of SQLite
SQLite’s design choices make it well-suited for some scenarios but poorly suited for others. Here are some places where SQLite doesn’t work well:
Apps that use features SQLite doesn’t support: SQLite does not support—and in many cases will not try to support—various relational database features. Many are corner cases, but even one of those can break the deal.
Apps that require scale-out designs: SQLite instances are singular and independent, with no native synchronization between them. They can’t be federated together or made into a cluster. Any software application that uses a scale-out design can’t use SQLite. As noted above, some third parties have extended SQLite to add those features, but they’re not native to SQLite’s design.
Apps with simultaneous write operations from multiple connections: SQLite locks the database for write operations, so anything involving multiple simultaneous write operations could result in performance issues. Apps with multiple simultaneous reads are generally fast, though. SQLite 3.7.0 and higher provide Write-Ahead Logging mode to make multiple writes work faster, but it comes with some limitations. For an alternative, considered Berkeley DB, mentioned above.
Apps that need strong data typing: SQLite has relatively few data types—no native datetime type, for instance. This means that the application must enforce most types. If you want the database, as opposed to the application, to normalize and constrain inputs for datetime values, SQLite may not work for you.
Microsoft released multiple updates to its database offerings at its Build 2024 conference.
One of the major updates to its database offerings includes the addition of vector search to Azure Cosmos DB for NoSQL.
Azure Cosmos DB for NoSQL, which is a non-relational database service, is a part of the larger Azure Cosmos DB database offering, a distributed database which implements a set of different consistency models enabling users to trade off performance against latency in their applications.
Fundamentally, NoSQL databases do away with SQL databases’ constraints of data types and consistency in order to achieve more speed, flexibility, and scale.
Azure Cosmos DB supports working with different data models, including APIs for MongoDB and Apache Cassandra. The database also has a flavor of PostgreSQL.
Last year at Build, Microsoft had introduced vector search to Cosmos DB, building on the company’s Cosmos DB for MongoDB vCore service.
The vector search in Cosmos DB for NoSQL, according to the company, is powered by DiskANN—a suite of scalable approximate nearest neighbor search algorithms that support real-time changes.
In addition, the company also made the Azure Database for PostgreSQL extension for Azure AI generally available in order to bring AI capabilities to data in PostgreSQL.
“This enables developers who prefer PostgreSQL to plug data directly into Azure AI for a simplified path to leverage LLMs and build rich PostgreSQL generative AI experiences,” the company said in a statement.
Additionally, the company said it is working to add an embeddings generation feature inside its Azure Database for PostgreSQL offering. The new feature, which is currently in preview, can generate embeddings within the database.
Further, the company said it was adding more capabilities to Copilot inside databases to help developers, including adding the ability to provide summaries of technical documentation in response to user questions inside Azure Database for MySQL.
EDB
