IBM Research added to its growing family of "trusted AI" tools recently with the release of a new open-source developer toolkit called Uncertainty Qualification 360 (UQ360). The new toolkit focuses on what IBM believes will be the next big area of advancing trust in artificial intelligence: communicating an AI's "uncertainty."
Uncertainty quantification is just what it sounds like: a determination of the level of confidence an AI system has in its decisions. The new UQ360 toolkit was designed to give data science practitioners and developers a set of algorithms to streamline the process of quantifying, evaluating, improving, and communicating uncertainty of machine learning models.
What we're talking about here, IBM AI researchers Prasanna Sattigeri and Q. Vera Liao explained in a blog post, is a way to enable an AI system or application to express that it is unsure, "giving it intellectual humility and boosting the safety of its deployment."
IBM is billing UQ360, which was released at the 2021 IBM Data & AI Digital Developer Conference, as one of the first toolkits designed to provide both a comprehensive set of algorithms for quantifying uncertainty and the capabilities to measure and improve uncertainty quantification to streamline the development process. The tool comes as a Python package with a taxonomy and guidance for choosing these capabilities based on a developer's needs, the company says.
UQ360 is just the latest toolkit to emerge from IBM Research, alongside AI Fairness 360, the Adversarial Robustness Toolbox, AI Explainability 360 and AI Factsheets 360, all released over the last few years to advance various dimensions of AI trust.
"Trust" in this context refers to the ability of humans to have confidence in the output of an AI-enabled app or system. AI systems have traditionally been black boxes, but, as IBM puts it, "To trust a decision made by an algorithm, we need to know that it is fair, that it’s reliable and can be accounted for, and that it will cause no harm." That level of trust requires transparency.
The fatal highway crash of a Tesla vehicle operating in self-driving mode in June threw another spotlight on the AI safety issue and the growing interest in shining a light in the AI black box. But Sattigeri, with whom I spoke over Zoom, said "miscalibrated uncertainties" are about more than just this kind of obviously critical application of AI.
"The self-driving example is a scary one," he allowed, "but take the loan approval process, where somebody is using an AI system to assist them in making a prediction that impacts your interest rate. Or in a healthcare setting, where the doctor needs to trust the AI to assist in making a diagnosis."
Quantifying uncertainty can show gaps in the knowledge of the training model, Sattigeri said, so the model can be improved.
"If we know [that the systems] are overconfident or underconfident," he said, " we can use recalibration algorithms to make them either loser, so you're increasing the margin of error, or [tighter] so you're decreasing the margin of error. And then it's up to the decision maker how they want to use it. If the uncertainty is too large, the loan officer can go ahead and do certain other investigation, maybe collecting addition information about the person."
If you've never visited Big Blue's R&D division website, you've just gotta. On the Trusted AI page alone, you'll find projects ranging from AI Explainability to Adversarial Robustness, Casual Inference to AI Fairness—all concepts behind research projects leading to the development of tools "to make AI more explainable, fair, robust, private, and transparent," IBM says.
AI software development continues to be a land of evolving concepts and esoteric nomenclature that coders with little to no experience in this terrain are increasingly required to navigate. But even AI road warriors need effective tools to keep up with the accelerating pace of software delivery that increasingly includes AI, machine learning, and deep learning. With its open-source trusted AI toolkits, IBM has put up some useful signposts.
Posted by John K. Waters on August 6, 20210 comments
Microsoft has amped up its support of Java developers by expanding its participation in the Eclipse Foundation to become a Strategic Member, the company announced this week.
Microsoft's Stephen Walli, principal program manager in the Azure Office of the CTO, will be joining the foundation's board of directors.
"The Eclipse Foundation is expanding its role through working groups and many of these working groups are important to Microsoft and its partners," Walli said in a blog post. "Recent work around the Eclipse Dataspace Connector and Eclipse Tractus-X are examples of new work beginning at the Eclipse Foundation in working groups in which Microsoft has an interest in participating."
Among other privileges, Strategic Members have a seat on the foundation's board of directors, its architecture council, and expanded board voting rights on key aspects of the Eclipse ecosystem, including licensing, governing policy development, and amendments to membership agreements and bylaws."
"Strategic Members play an integral role in the Eclipse Foundation ecosystem," the foundation explains on its website, because they are "investing significant developer and other resources to further drive Eclipse Foundation technology."
How much of an investment? According to the foundation's membership agreement, Strategic Members must commit "the full-time equivalent" of at least two developers assigned to work on Eclipse technology projects on an on-going basis. Strategic Members are encouraged (but not required) to lead an Eclipse project or a Project Management Committee (PMC). They also pay dues based on a combination of membership class and the organization's annual revenues. A company making more than €250M (just under $3M) annually pays €250,000 for a Strategic membership. (You can check my math on the membership agreement.)
"The Eclipse Foundation has a long history of providing a strong, collaborative culture supporting open-source-licensed projects," Walli said in his post, "and many of those projects are important to Microsoft, our partners, and our customers. It is important for Microsoft to support the organization that supports those projects, and to work within the organization towards those collective goals."
Another reason for Redmond's accelerated involvement: the foundation's recent decision to establish its official headquarters in Belgium.
"The [Eclipse] team showed initiative and forethought and pivoted to become a European-based international non-profit organization to align with its membership," Walli said in his post. "The Eclipse Foundation is a natural place for Microsoft to collaborate on new initiatives beginning with European partners."
Walli posts occasionally on Microsoft's Open-Source Blog. He has a long history of working with open source.
"Having a rich ecosystem of healthy non-profits supporting different groups of open-source-initiative-licensed projects and their project ecosystems is a must," Walli said. "At Microsoft, we are committed to continuing to support and participate across the non-profit ecosystem, as well as engage in projects themselves."
Of course, Microsoft supports its own thriving developer community, but Redmond has been a member of the Eclipse Foundation since 2016, when it joined as a Solutions Member. The company announced the Microsoft Build of OpenJDK, a new no-cost, open-source, Long-Term Support (LTS) distribution of OpenJDK, in April of this year, and released it in May. And Java on Visual Studio Code has become an increasingly popular code editor.
But Microsoft has seemed especially interested in supporting Java on Azure. The Azure Toolkit for Eclipse, which is available on the Eclipse Marketplace, provides "functionality that allows you to easily create, develop, configure, test, and deploy lightweight, highly available and scalable Java web apps and HDInsight Spark jobs to Azure using the Eclipse development environment. The Java SDK for Azure is an open-source Azure SDK for Java designed to simplify provisioning, managing, and using Azure resources from Java application code.
The Eclipse Foundation is a good place from which to pursue at least one dimension of these interests. It's one of the world’s leading open-source software development and specifications organizations. It's the non-profit steward of the Eclipse IDE, enterprise Java (Jakarta), and the Eclipse MicroProfile. And its roster of Strategic Members includes IBM, Oracle, Huawei, and SAP, among others—now including Microsoft.
Posted by John K. Waters on August 4, 20210 comments
The Eclipse Foundation today announced the formation of the Eclipse IDE Working Group, a new community-driven initiative that will "support the continued evolution, adoption, and sustainability of the Eclipse IDE suite of products, related technologies, and ecosystem."
Specifically, the new working group will provide governance, guidance, and funding for the communities that support the delivery and maintenance of Eclipse IDE products. The stated goals of the group are "to ensure the continued success, vibrancy, quality and sustainability of the Eclipse Platform, desktop IDE and underlying technologies, including related planning and delivery processes, as well as related delivery technology."
The animating idea here is to provide a governance structure that will enable broad collaboration while maintaining standards and addressing market requirements. That structure will be supported by an impressive list of working group founders that includes Bosch, EclipseSource, IBM, Kichwa Coders, Renesas, SAP, VMware, and Yatta Solutions.
The Eclipse IDE was not only the Foundation's flagship offering when the organization was created by IBM and set loose upon the world in 2001, it was its only offering, its raison d'ê·tre. Looking back to my first report on the then-fledgling dev tool, I found this description: "The Java-based, open-source software, code-named Eclipse, will enable developers to use tools from multiple suppliers together, allowing them to integrate processes used to create e-business applications, such as those for Web services." (Code-named? E-business?)
When IBM announced that it was releasing its Eclipse code, Gartner analysts Joseph Feiman and Mark Driver called the move "an ambitious project and an ambitious product foundation." If it succeeded, they said, it would revive the concept of best tools combined in a single workbench, which they called "an application developer's dream."
It would be an understatement to say that the open-source community—heck, the tech world at large—viewed IBM's largesse at the time with skepticism. And yet, Big Blue managed to keep from big-footing the Foundation, participating, instead, as a regular-sized-foot member organization.
"Two decades ago, IBM and the community launched what has become the Eclipse IDE family of projects, and these tools are even more useful to developers today as they were then," said Todd Moore, VP of IBM's Open Technology, group, in a statement. "As a founding member of the Eclipse IDE Working Group, IBM is eagerly looking forward to collaborating with the other members, supporters and the community to nurture a thriving ecosystem that keeps these projects relevant now and into the future."
Since then, the Foundation has evolved into a steady, competent specification organization and the true shepherd of a vast expanse of solutions and services. And ever at the heart of that expanse has been its venerable, namesake IDE. With millions of users, tens of millions of downloads, and billions of dollars in shared investment, the Eclipse IDE continues to be one of the most popular desktop development environments on the planet.
"For 20 years, the Eclipse IDE has provided developers around the world with a powerful open-source tooling platform used to create world-class applications and products," said Mike Milinkovich, executive director of the Eclipse Foundation, in a statement. "This new working group will ensure the Eclipse IDE platform is set to fulfill the needs of the millions of developers who use the Eclipse IDE today and in the future."
The launch of the Eclipse IDE Working Group coincides with the 2021-06 release of the Eclipse IDE, itself, and the second quarterly simultaneous release of more than 70 participating projects, 110 committers, 174 contributors, and almost 80 million lines of code, the Foundation says.
A partial list of new features for this release includes:
- Support for Java 16 and the necessary tooling for development
- Improved Java development tooling with new cleanups added, improved debug hover, and evaluation over chain of variables
- Support for Mac AArch64 for Apple M1 based systems
- Improved embedded terminal that supports opening files and links with Ctrl+Click, remembers working directory, shell, and other settings
The Eclipse IDE 2021-06 is available now for download.
Unsurprisingly, the Eclipse Foundation is welcoming interested parties to the new working group. To learn more about how to get involved with the Eclipse IDE Working Group, visit the Eclipse Foundation membership page or see the working group’s Charter and Participation Agreement.
Posted by John K. Waters on June 17, 20210 comments
Lightbend, the company behind the Scala JVM language and developer of the Reactive Platform, today unveiled "a unique, first-of-its-kind Platform-as-a-Service (PaaS) offering" for cloud native application development.
I put quotation marks around that marketing phrase (it's a reflex), but the newly available Akka Serverless does look to be a unique offering.
Akka Serverless is based on a new architecture that removes limitations that have, historically, prevented the development of general-purpose applications using the serverless model. With those limitations removed, the new PaaS enables the creation of cloud native apps using any programming language and eliminates the need for databases in deploying business-critical applications.
Brad Murdoch, Lightbend's EVP of Strategy, talked with me about the concept of a "stateful serverless" developer platform.
"The challenge with stateless, of course, is that, to build certain classes of applications, you need state," Murdoch said. "And you need state management at scale, and you need state to be a first class citizen, and you need your data available in real time, and all of those great things that are basically not possible to do in a stateless serverless model today.
"We're changing that, but we're not just sort of adding state to serverless. We've built a declarative API-first programming model where the developer basically defines the API contract and the data they want their function to be able to operate on at runtime. Then Akka, under the covers, delivers that data to the function at runtime automatically."
What this does, effectively, is to remove whole swathes of middleware and database concerns from the developer, Murdoch said. And with Akka "under the covers," this model scales, so the developer doesn't need to worry about that, either.
"The big change here is that we're kind of reversing the paradigm, so that instead of the developer needing to understand the complexities of distributed computing in order to take advantage of it for scale and performance, now we've got a service on the cloud that can do all that for you," Murdoch added.
Akka Serverless also delivers on a goal of the company that dates back to its origins as Typesafe: it is a fully polyglot platform that can support any programming language that can talk to the API.
The other big change with this announcement is that Jonas Bonér, Lightbend’s CTO and founder, inventor of the Akka Project, and co-author of the “Reactive Manifesto,” has also taken the helm of Lightbend as CEO.
“Until now, the challenges of dealing with state at scale have meant that a serverless approach has not been able to address many more complex applications, such as IoT platforms, real-time financial services, modern eCommerce systems, streaming media, internet-based gaming, factory automation, telemedicine, and more,” Bonér said in a statement. “What is needed is a data-centric backend application architecture that can handle the volume of data required for today’s applications at extremely high performance. Akka Serverless is the first to achieve these capabilities and I’m excited to continue Lightbend's growth leveraging this superior model for cloud native development.”
Bonér, a Swedish programmer who had built compilers, runtimes, and open source frameworks for distributed applications for vendors such as BEA and Terracotta, was frustrated by the scale and resilience limitations of CORBA, RPC, XA, Enterprise JavaBeans, SOA, and the Web Services standards and abstraction techniques Java developers used at the time. He turned to the Actor Model, which emphasizes loose coupling and embracing failure in software systems and dataflow concurrency, used by the Erlang and Oz languages.
Bonér developed the concept of the Akka Actor Kernel (later shortened to Akka) in 2009 and shared the first public release of Akka 0.5 on GitHub. It would eventually become the de facto model for concurrency in the Java Virtual Machine (JVM). Today Akka is used by PayPal, LinkedIn, and Capital One, among many others, to handle billions of transactions at massive scale in their cloud-native systems. It’s also used by the popular online game Fortnite.
Akka Serverless was two years in the making, Murdoch said, and is now available in open beta, with general availability expected later this year. Developers can learn more about Akka Serverless and become a beta participant today.
Posted by John K. Waters on June 10, 20210 comments
There are so many announcements coming out of this year's Google I/O virtual conference this week that I couldn't cover it all it in one post. So, here's a quick rundown of other announcements from this (very well produced) event. (Loved the outdoor keynotes.)
Google's Jacob Lehrbaum, director of Android developer relations, advised Android developers to get ready for Android 12, due later this year, but available now in beta for some devices. This release will come with "one of the biggest design changes ever" in Android's history, he said. And that's not an understatement.
The stunningly refreshed UI includes a new "color extractions" feature that changes the system’s theme based on your current algorithm, a dynamic lighting feature, and a Quick Wallet Access feature on the lock screen.
Android 12 also comes with some new safety features, including the ability audit app data requests, which is similar to the privacy features in the recently updated iOS 14.5. Developers can learn more about how their apps and their dependencies access private data from users by performing data access auditing. (Details on this Android Developer page.)
The Android 12 beta is available to anyone with a Pixel 3 or newer, the Xiaomi Mi 11 and devices from ZTE, Asus, OnePlus, Oppo, realme, Sharp, TECNO, Vivo, and TCL. The Nokia X20 is also on that list, though no link was available on the Android 12 'eligible devices' page at press time.
Google also announced new features for its Firebase mobile web app development platform "after taking a hiatus in 2020." The platform, which Google says is now used by more than 3 million apps per month, gets a new AI-powered Personalization enhancement of its Remote Config service. Currently in alpha, Personalization uses Android's on-device machine learning (ML) capabilities to allow devs to deliver optimized individual user experiences automatically. The only thing current Firebase devs will need to do is provide different configuration options and Personalization will monitor app users and tailor aspects of those configuration options to what it "thinks" will improve engagement.
Google also announced that devs can now use the Firebase Storage API locally through the Emulator Suite, a feature that lets developers run local versions of some of the tools in Firebase. The Firebase Local Emulator Suite is a set of advanced tools for developers "looking to build and test apps locally using Cloud Firestore, Realtime Database, Cloud Storage, Authentication, Cloud Functions, Pub/Sub, and Firebase Hosting," the company says.
Anything that uses this API can now be tested completely offline, which means devs can take advantage of cost savings and enhanced testing automation.
Android 12 also comes with new hardware APIs for Chrome designed to provide access to device peripherals, and new Privacy Sandbox APIs now available to test. Remember that Google plans to phase out third-party cookies from Chrome.
And then there's Flutter, version 2.2 of which was announced at the conference. This release of Google's open-source mobile UI toolkit comes with several new features, including a way for developers to monetize their apps with in-app purchases and ads. This release also makes it possible for developers to connect to cloud services, and it comes with APIs that extend apps to support new capabilities.
Based on Google's in-house language, Dart, Flutter was released by Google in 2017, and it's star has been on the rise ever since.
Google also launched a very cool new MLOps platform called Vertex AI, which you can read about in our sister publication, Pure AI.
Posted by John K. Waters on May 19, 20210 comments
Google kicked off its 2021virtual I/O conference this week with a multiple-camera keynote staged outside on the Googleplex campus in Mountain View, CA, that included a number of announcements for developers.
Topping the list of dev-related announcements at this year's event: the latest version of the Android Studio IDE, Arctic Fox (2020.3.1), gets released to beta. Among the many updates and enhancements in this release, the standout is the integration of Google's native UI toolkit Jetpack Compose, the 1.0 release of which is due in July.
Jetpack Compose is a Kotlin-based declarative design framework for Android, desktop, and the web. (Desktop and web development is managed by the IntelliJ IDEA IDE.) It's used to create previews in different configurations and allows developers to navigate their code with Compose Preview, test it in isolation with Deploy Preview to Device, and inspect the complete app with Layout Inspector. Throughout iterations, devs can quickly edit strings and numbers and see immediate updates. Plus, with the Accessibility Scanner in Layout Editor, the View-based layouts are audited for accessibility problems. Jetpack Compose is designed to work with existing code and can be added to existing apps.
Google announced the alpha release if Jetpack Macrobenchmark, which helps developers analyze startup and animation lag in apps and pinpoint related issues.
Kotlin itself got a nod at the conference, including the announcement of a stable release of a new annotation processor API, called Kotlin Symbol Processing, which Google said should be as much two times faster than previous options, and generally more efficient. And the Kotlin DataStore, a Kotlin-first alternative to SharedPreferences, has entered beta.
The Arctic Fox release also adds support for Live Editing of literals, which means developers using Compose can edit literals (strings, numbers, Booleans) in their code and see the results immediately without needing to wait for compilation. "The goal of the feature is to increase your productivity by having code changes appear near instantaneously in the previews, emulator, or physical device," explained Paris Hsu, interaction designed from Google's Android product and design group, explained in a blog post.
There's also new support for Google's Wear OS, Google TV, and Android Auto via new emulators and system images, not to mention the Samsung Galaxy Z Fold 2.
And there's a list of new features and improvements in Arctic Fox that come with a major update of the IntelliJ IDEA IDE (v2020.3), which allows developers to test their apps with new features in Android 12, improve app performance with the updated UI for Memory Profiler, understand background task relationships with WorkManager Inspector (which allows devs to schedule deferrable, asynchronous tasks that must be run reliably), and use Non-Transitive R classes IDE Refactoring to increase build speed.
Speaking of IntelliJ, the Arctic Fox beta includes the IntelliJ 2020.3 platform release; you'll notice Arctic Fox 2020.3.1 employs a similar numbering system. Hsu explained in her post:
As we announced late last year, we've changed our version numbering scheme to match the number for the IntelliJ IDE that Android Studio is based on, 2020.3, plus our own patch number, as well as a handy code name to make it easier to remember and refer to. We'll be using code names in alphabetical order; the first is Arctic Fox, now in beta, and the next is Bumblebee, now in canary.
The beta of Android Studio Arctic Fox is available now for download, as is the canary build of Bumblee (2021.1.1) here.
Posted by John K. Waters on May 19, 20210 comments
Open-source Java development tools and runtimes provider Azul has announced plans to unite its commercial products into a single "Java platform for the modern cloud enterprise." Called the Azul Platform, it bundles the company's Zulu builds of OpenJDK and its Zing Java Virtual Machine (JVM), under a new set of services called the Azul Intelligence Cloud.
Users of the company's products will be able to develop, deliver, optimize, and manage their Java applications via this new platform, the company says.
"What we're doing with the platform is providing a more holistic vision of Java and the enormously popular OpenJDK," Azul CEO Scott Sellers told me. "There's a lot more to Java these days, with cloud and hybrid cloud deployments taking it to a whole new level. Java was architected more than 25 years ago for a different day and age and a different set of metrics. The higher-level capabilities we're providing we think are critical for Java's continued success. And we're providing them in a very developer-friendly, open-source manner consistent with ongoing community efforts around OpenJDK."
Sunnyvale, Calif.-based Azul bills itself as the only vendor focused exclusively on Java and the JVM. It's Zulu builds of OpenJDK are tested and certified open-source Java Development Kits (JDKs) that are free to download and use without restrictions, with optional support available via subscription plans. Its Zing "no pause" JVM is based on Oracle's HotSpot, a core component of Java Standard Edition.
Clearly, there's a lot of rebranding going on with this announcement. In the context of the new platform, "Zulu Enterprise" and "Zulu Embedded" are now "Azul Platform Core." And "Zing" is now "Azul Platform Prime." And the company has a new logo. But it's also true that Azul is uniting it's sturdy product catalog into a platform with additional features and capabilities enabled by the changes—not to mention a new solution that will pull it all together: The Intelligence Cloud.
This latter, pending addition to the product lineup is the most intriguing part of the announcement, because, among other things, Zulu and Zing are well known products, and the Intelligence Cloud won't actually be available until the second half of 2021.
The new Azul Intelligence Cloud comprises three components:
- The Azul Analytics Suite, an analytics platform designed to provide "actionable operational intelligence" around artifact inventory management, security, and anomaly detection.
- The Azul Optimizer Suite, made up of "optimization" modules for Azul’s builds of OpenJDK that enable "increased levels of performance and throughput completely transparent to the application code."
- Connected Runtime Services (CRS), which connect Azul runtimes to facilitate communication with the analytics platform and optimization engines.
By uniting Zing and Zulu under the Intelligence Cloud, the company is aiming to provide a new level of "visibility, management, security, stability, efficiency, optimization and operational intelligence across entire Java estates."
"This broader platform offering is what we think it takes to as serious enterprises look to evolve their Java strategies," Sellers said. "Sun, and now Oracle, have done right by them for a long time, and we believe we're evolving into a more interesting and exciting future. But that future requires a broader vision, and we're providing that with the new platform."
The company promises delivery of the new platform later this year.
Posted by John K. Waters on May 12, 20210 comments
Oracle Linux users in North America are gathering online tomorrow (Thurs. May 6, 10am PT) for the latest edition of the State of the Penguin. Wim Coekaerts, Oracle Software Development SVP and Linux Foundation Vice Chairman, will be leading what promises to be an enlightening conversation about the industry landscape, customer use cases, and the latest Oracle Linux technologies, including containers, KVM, open-source contributions, and developer tools, all to help Penguinistas "explore possibilities and update your plans."
Coekaerts' co-host for the event will be Sergio Leunissen, VP in Oracle’s infrastructure engineering team. Leunissen currently leads initiatives to deliver solutions for developers on Oracle’s operating system and Oracle Infrastructure Cloud, and he’s responsible for Oracle’s presence on GitHub.
I had the opportunity to talk with Coekaerts about the event last week. He's widely described as an "industry luminary," an appellation I found to be something of an understatement. He led the last online State of the Penguin, held six months ago.
"We're trying to provide an update on a regular basis to users of Oracle Linux, specifically, but also Linux in general," Coekaerts told me. "We want to let the community know what we're doing, and the 'state of the penguin' was just sort of a good headline for that."
The event will take the form of an informal Zoom chat, Coekaerts said, without the usual structured Power Point presentations offered by most webinars (though there will be some slides). It's planned as a true Q&A session, reminiscent of the original "state of" events at the annual Oracle OpenWorld conference.
The event definitely gives Big Red a stage to blow its own horn about its contributions to the Linux community. Coekaerts reminded me that Oracle is consistently among the top three or four contributors to the Linux kernel, that the company is consistently adding features and lots of lines of code, all of which is readily available for code review.
"People don't realize how many code reviews we do," he said, "and they sometimes forget that doing code reviews and sign-offs is a big part of why Linux is as good as it is. Everyone looks at everyone else's code and provides comments and they formally say, I put my name on this, I looked at it and I agree that it's good code. A lot of that stuff is done by the people in our team."
I asked Coekaerts what was top-of-mind for him as the "state-of" event approaches. He pointed to the work his company has done on Linux on ARM-based chips. "We put a lot of effort into making Linux run really well on ARM on the server," he said, "and we've been applying all the knowledge we've gathered over the years on how to make a good server platform from an operating system point of view. We're taking that knowledge and basically applying it to ARM, because, although it has been around a long time for 32-bit servers, it has not for 64-bit servers."
He also pointed to the work Oracle has done on security around the QEMU hosted virtual machine monitor.
"The way QEMU works, it's one process that runs along with the VM," he explained. "And so there has been a concern that if the VM has a sort of backdoor into QEMU, you get to the host and there's a control platform. We have done a huge amount of work separating that, so there's one process running for managing QEMU, and another process that actually contains the stuff the VM needs. We've isolated the address space, and that's very important from a security point of view. And there's nothing in this that's specific to Oracle. Everyone else can make use of it."
Coekaerts' team is also working on a new memory access strategy that takes into account the drive toward higher core counts, and the resulting contention on locks in the kernel.
"What tends to happen as the cores grow is that people run more processes, so there's more contention on locks in the kernel," he explained. "So, as we run the database, for example, we discovered that there are significant performance impacts on getting access to virtual memory."
Coekaerts' team also came up with a new way of dealing with memory access called "Maple Tree."
"It went upstream into the kernel," he said. "I think it's a good example of how we try to make Linux better, in general."
Coekaerts felt it was important to mention that the complete Oracle Linux kernel, with the complete change logs starting from Linux 3.0, are available on GitHub. "Every change we make, all the errata, every bug fix—it's all publicly out there," he said.
With the release of Oracle Linux 8.4 on the horizon, attendees can also expect to get a peek at the roadmap, Coekaerts said, including an update on the Ansible product, Oracle Linux Automation Manager, the upcoming Oracle Cloud Native Environment 1.3 (basically an uptake of the latest version of Kubernetes), and upcoming releases of Unbreakable Enterprise Kernel (UEK) for Oracle Linux.
There's more on the agenda, and Coekaerts had a lot to say, but I don't want to give away any plot twists. Anyone can register to attend the event.
Posted by John K. Waters on May 5, 20210 comments
The challenges of building applications that need to handle the massive volumes and countless sources of time-stamped data produced by sensors, applications, and infrastructure are myriad. Because of the uniquely critical need for efficient communication among dev team members working with what is known as "time series" data is critical, distributed teams especially challenged.
Enter time series database provider InfluxData, which recently announced a solution to this challenge, at least within the InfluxDB Cloud. The new InfluxDB Notebooks allows developers to discuss time series data analyses and trends inside the platform, so team members don't have to use third-party messaging apps, which can slow them down significantly. This new capability allows users to create what the company calls "a durable artifact" that shows teams how time series data is analyzed to solve business problems.
"Development teams are more distributed than ever, but until now, they haven’t had the tools they need to seamlessly communicate around time series data," said Russ Savage, director of product management at InfluxData, in a statement. "To solve this problem, we’ve reimagined InfluxDB as a way to collaborate around data, not just store it. This new approach will dramatically save time for developers, so they can focus on building software."
InfluxData provides a time series database platform aimed at developers building Internet of Things (IoT), analytics, and monitoring software. The company's new InfluxDB Notebooks adds a number of capabilities to the platform, including
- Design time series data pipelines with dynamic data, live code, and real-time visualizations — all with inline explanatory notes. This capabilities allows dev team members to show their work and sharing it with others.
- The ability to share incident investigations to explain root causes following service outages, and to build "runbooks" to avoid future outages.
- The ability to document how Internet of Things sensor data has been collected, normalized, enriched, and "downsampled" to facilitate preventive maintenance and forecast device obsolescence.
The company says it plans to add another new capability to its platform: InfluxDB Annotations, which will make it possible to add notes directly on dashboard cells to more quickly highlight and explain the meaning of anomalies in time series data, and to coordinate troubleshooting efforts. It will be used to capture context and share details about ongoing investigations into outlier data points that are underway.
This capability will save teams time "by eliminating the need for multiple people to repeat the same investigation," the company says. Both features are designed to support "better collaboration workflows between developers, SREs, and every stakeholder involved in time series collection, enrichment, and analysis."
Where InfluxDB Notebooks allows teams to "weave together computational information" such as code, data, and statistics with narrative and graphs, InfluxDB Annotations will help developers "share contextual clues" so they can quickly determine the root cause of incidents and restore services faster.
Posted by John K. Waters on April 29, 20210 comments