Open Source Developer Feuding Gets Uglier

I admit, I thought some of Linux creator Linus Torvalds' rants in his public feuds with other open source developers were kind of funny, even if some were over the top. His style rubbed off on others, generating several contentious back-and-forth exchanges on mailing lists that I still found humorous sometimes.

Death threats aren't funny, though.

The squabbling has reached new, ugly heights as revealed by Red Hat Inc. developer Lennart Poettering in a Google Plus post earlier this month in which he related threats of violence and even a movement to hire a "hitman" to deal with him.

"Much of the open source community tries to advertise the community as one happy place to the outside," Poettering said. "Where contributions are valued only by their technical quality, and everybody meets at conferences for beers.

"Well, it is not like that. It's quite a sick place to be in."

Poettering, who worked on the systemd component of Linux, placed most of the blame for this "sickness" on Torvalds. "By many he is a considered a role model, but he is quite a bad one," Poettering said.

Systemd, as described by Wikipedia, "is a system management daemon designed for Linux and programmed exclusively for the Linux API. For systems using it, it is the first process that is executed in user space during the Linux startup process."

For whatever reasons, systemd has generated quite a bit of animosity among other developers, Torvalds included. A few years ago there was even a change.org petition set up to compel Poettering to "Stop writing useless programs," including systemd. It was closed with 235 signatures.

On Hacker News, his post generated 169 comments. They were all over the map of course, but even on that site some extreme vitriol emerged, with one commenter calling him "a scourge and a scum" and another saying he "is a complete ___hole and deserves all the hate he gets."

What's going on here? It's only software, folks.

It even gets worse, as Poettering explained:

Recently, people started collecting Bitcoins to hire a hitman for me (this really happened!). Just the other day, some idiot posted a "song" on Youtube, a creepy work, filled with expletives about me and suggestions of violence. People post Web sites about boycotting my projects, containing pretty personal attacks. On IRC, people /msg me sometimes, with nasty messages, and references to artwork in 4chan style. And there's more. A lot more.

Frankly, I don't want to hear anymore. The humor is gone and there are a lot more serious things going on in the world to get upset about. Poettering said he is done with it, too.

"I am not the one to fix any of this, I cannot tell you how one could do it," he said about the open source infighting. "And quite frankly, I really don't want to be involved in fixing this. I am a technical guy, I want to do technical things."

Finally, he concluded: "And that's all about this topic from me. I have no intentions to ever talk about this again on a public forum."

Posted by David Ramel on October 16, 20140 comments

Internet of Things Takes Center Stage in App Development

Up 'til now it's been a supporting character in the ongoing "Hypes of Our Lives" drama starring Big Data, cloud computing and mobile-first, but a flurry of announcements this week confirms the Internet of Things (IoT) is striving for top billing.

The IoT was the feature attraction at this week's Intel Developer Forum in San Francisco. In announcing new developer tools, the hardware giant highlighted "how the company is moving quickly to enable new market segments where everything is smart and connected."

Among those developer tools was the Analytics for Wearables program to boost the creation of wearable apps with data-driven intelligence, free of charge. It was part of the company's new direction, exemplified by how this year's conference "expanded beyond PCs, mobile and the datacenter to also include the IoT and wearables and other new devices." The conference even featured an "IoT mega session."

Intel's Doug Davis yesterday started out a related blog post by declaring, "There's no doubt about it: The IoT is influencing and changing the way the world works -- from new opportunities to collect and analyze Big Data in industrial settings to advances in autonomous vehicles and everything in between."

In that IoT mega session, Davis detailed 10 Intel architecture-based "unexpected IoT applications," including keeping tabs on endangered rhinoceros by fitting them with ankle bracelets. (I'm right now starting a rumor that Apple will announce a "rhino watch" in an upcoming, breathlessly awaited, mysterious and dramatic stage show. My sources say a rhino will trot out amid cool thunderous music to unveil it, wearing a black turtleneck.)

I didn't watch this week's "unveiling" of Apple's wearable, but dozens of headlines dutifully reported it will play a big role in the IoT, so expect a lot of developer news about building apps for it.

Dell partnered with Intel to make its own news this week by opening the Dell IoT Lab, partially funded by Intel. It will provide space for building, testing and modifying new IoT solutions. In the Dell announcement, Intel's Davis said, "The IoT will enable one of the most significant technological transformations in our lifetime." Sound familiar?

Embarcadero got in on the act with this week's announcement that "Developing Connected Apps for the IoT Just Got Easier with Appmethod." The updated Appmethod development tool "provides a complete software development solution for building natively compiled C++ apps that connect mobile and desktop devices, sensors, gadgets, wearables, cloud services and data."

Smaller companies also joined the party this week, such as Open Garden. The San Francisco startup on Monday announced a new Mobile Network for IoT devices. It will provide an API to access the network. "The Open Garden Network makes it easy for devices to seamlessly access other devices and the Internet via secure peer-to-peer connections," the company said. "Devices automatically detect nearby enabled smartphones or tablets via Bluetooth LE (Low Energy) and can pass data through them securely until the proper destination or recipient is reached."

"MachineShop emerged from stealth mode earlier this year to bring the power and simplicity of APIs and Web services to the realm of the IoT," the company announced. "Among the key enhancements to the MachineShop Connected Services Platform is the addition of nearly 100 new APIs for data ingestion, normalization and management."

And, to help developers make money with this new roadshow, Aria Systems got a jump on everybody last week by announcing an e-book about "Monetizing the IoT."

Forbes quickly hopped on that bandwagon yesterday with an article proclaiming "How Investors Can Profit from the IoT."

Every new flavor-of-the-week industry disruption needs an app store, of course, so on Tuesday came the news that "RacoWireless Enables Rapid IoT Deployments with Launch of IoT App Store."

"The Application Marketplace allows application developers to create new revenue streams by offering their value-added products and features to a growing community of solution and service providers," the company said.

You get the picture. If you're one of those underpaid developers who signed on to get a data science degree to make bank with Big Data, you'd better add an IoT minor. Of course, the IoT is closely related to the tech industry's current headliners -- Big Data, mobile computing and the cloud -- so jumping on the IoT bandwagon shouldn't pose too much trouble in transferring skills.

You also might want to bone up on your IP chops -- specifically IPv6. We've been warned for years that we're running out of IPv4 addresses, and this IoT boon isn't going to help any. "The public IPv4 address space managed by IANA has been completely depleted by Feb. 1, 2011," reads and IEEE position paper from last year. "This creates by itself an interesting challenge when adding new things and enabling new services on the Internet."

No kidding. Last month, an Internet slowdown was blamed on old IPv4 router settings, which could be a harbinger of problems to come once all those cupcake dispensers, toilets and beer kegs are connected to the 'Net (those were some of Intel's other unexpected IoT apps).

Anyway, just be thankful you're sitting at a desk typing for a living, instead of fitting wild rhinos for ankle bracelets.

Posted by David Ramel on September 11, 20140 comments

On the Death of Big Data

Big Data died when I wasn't paying attention.

Recalibrate the hype machine: It's "all data" now. I learned this while talking to industry experts to research an upcoming article on SQL Server 2014 and how it fits into the new-age world of what-used-to-be-called-Big-Data.

Analyst Donald Feinberg at research firm Gartner Inc. doesn't even like to use the term. Don't get him started.

"'Big Data' is going to disappear in the next two or three years and it's going to be called -- guess what? -- 'data,'" Feinberg told me. "Once every vendor has a Big Data something, it no longer gives me a way to distinguish one vendor from another."

I also talked to David Torres, senior director of Data & Analytics at Avanade Inc., which provides Microsoft-focused consulting services. "People ask, 'OK, what are you going to do with Big Data?’ Torres said. "I say, 'What are you doing with all data? What are you doing with your relational data? What are you doing with streams -- dark data? Are you taking advantage of all your data and really kind of using it as a corporate asset and trying to figure out how it can help you innovate your business?' Beyond Big Data, it's really, in my opinion, it's all data. You need to have a strategy for all your data."

Microsoft's strategy is to instill a "data culture" within an organization where everybody has access to vast new stores of analytical information of varying types and can use it to improve business processes and products. For what-used-to-be-called-Big-Data, Microsoft is providing several products and services for leveraging the traditional Apache Hadoop ecosystem while also taking advantage of the democratization of the data and increased access to it from technologies such as SQL. The company is actually baking a lot of the "BD-word" functionality into the company's flagship RDBMS, SQL Server 2014.

For Pivotal Software Inc., the strategy is a new term: "data lake."

"They don't talk about Big Data anymore, they talk about data lakes," Feinberg said. "It's something new. They coined the term."

Feinberg doesn't like that term, either -- "we hate it" -- though it does serve to distinguish Pivotal from other vendors in the space. "But the reason they coined that term, is because Big Data doesn't mean anything anymore, since everybody has something."

Another expert, Andrew Brust, doesn't seem so sure about Microsoft's term: data culture.

"Well, you hear lots of discussions about making businesses more data-driven, and I think that 'data culture' is kind of Microsoft's and [CEO Satya Nadella's] little variance of that same message," said Brust, a SQL Server MVP and research director at Gigaom Research. "I mean, it's real. If we can get businesses to the point where it's just a matter of course and practice that decisions are based more on data and metrics and less on hunch, that's I think an efficient thing and good thing. And it's obviously easier rhetorically than in practice to get there, but it's absolutely what's been driving all the changes in data in the last five years or so."

It certainly has been driving changes within Microsoft and SQL Server. Alex Barnes, a DBA on the front lines actually using SQL Server, noted this while talking about the 2014 version's new delayed durability capability, which concerns when a database write operation is acknowledged by the system. NoSQL databases play "fast and loose" with some of these traditional database rules for the sake of speed, said Barnes, who works at Charleston, S.C.-based BoomTown, a company that provides a platform for real estate professionals.

"SQL Server in 2014 has actually implemented an option to turn on delayed durability, which is basically, you could call it, like, 'NoSQL mode.' It's a way to say, 'I don't need write-ahead logging, I don't necessarily care if my system crashes and I lose a little bit of data. I'd prefer to have the speed up front. They've really kind of added these options in there to compete with these guys who are playing fast and loose with these rules. You can basically, now, by jamming a bunch of data into memory, creating these hash indexes, or range indexes, and turning on this delayed durability, you can basically turn SQL Server into what people think of traditionally as a NoSQL database."

Ted Neward, an expert database developer, author and consultant, noted how the NoSQL and traditional relational SQL worlds have been evolving and reacting to one another. The highly scalable NoSQL databases were an upstart technology taking on the traditional RDBMS vendors with new capabilities and agility. Neward noted how you can download, install and start running some NoSQL databases in less time than it takes to read about how to do that for SQL Server.

Microsoft has responded, Neward said, trying to provide some of the advantages of the NoSQL movement while not leaving its traditional customers behind. And now the "other guys" have responded in kind, opening up more to the SQL world.

"There was certainly a movement in the NoSQL world that basically said, 'dump your SQL databases, dump your relational databases -- they can't handle this new Internet world," Neward said. "Which I think was always a misnomer and always a mistake, and we see them changing their tune now, now that NoSQL supposedly stands for 'Not Only SQL'" instead of "No SQL."

"That's B.S.," Neward said. "That's revisionist history. We're not stupid."

Well, I may be stupid, but it's becoming increasingly clear that Big Data and "traditional data" are merging into one coherent, integrated data analysis concept. A quick Internet search shows terms like "Big Data is dead" have been out there for years now, propagated by forward-thinking pundits. Who knew?

Noel Yuhanna knew, for one. The analyst at Forrester Research Inc. is paid to know these things ahead of dolts like me. He agreed with Avanade's Torres about the need for organizations to adopt a coherent, overarching data strategy encompassing technologies of all types.

"Enterprises that have a more complete data platform story, as well as a vision, are more likely to succeed in the coming years and also have a competitive advantage if they get onto this bandwagon of data platform, which includes Hadoop, Big Data, NoSQL as well as traditional databases -- all integrated," Yuhanna said. "Because that's where you see customers that are more successful, having all those data types together and managed together and provided together in a manner that will be helpful for businesses to operate."

So, goodbye, "Big Data." R.I.P.

Now, about this "cloud" thing ...

Posted by David Ramel on September 8, 20140 comments

Again with the 'Codeless' Mobile App Building

Programming is hard. Programming modern mobile apps is wicked hard. There's no getting around it, though a lot of companies are trying to do just that.

Yesterday saw a new effort from CAIL Mobile Solutions. "Rather than going through the traditional app development process, the CAIL Mobile Design Platform provides the mobile enterprise with easy-to-use, codeless templates, prebuilt forms, drag-and-drop capabilities, and various setup options," the company said.

A new tool for the platform is the CAIL Mobile Doc-2-App tool for creating and sharing documents. Basically it takes documents in familiar formats -- Word, Excel, PowerPoint and PDF -- and converts them into mobile "apps" that can be viewed on iOS or Android devices. Another tool, Email-2-App, can work with the document tool to automatically convert documents attached to e-mail messages into apps, with no user intervention needed at all.

For app building, the platform lets users choose from different templates, such as classic, mosaic, list and full page, and then drag data icons into frames on the template to add text, images or more specialized functionality such as queries reporting "customers per quarter" or "orders per region" from a database. Once the app is edited and customized the way you want, you hit publish and users who have installed the app previously will see the updated content.

The system can be deployed on-premises behind a firewall or to private, public, hybrid or Software-as-a-Service (SaaS) clouds, with "no need for any coding or development to connect to your data sources."

Also yesterday came an announcement from another company wrestling with the document problem, Canvas Solutions Inc. It provides a cloud-based service to provide "an easy, fast and economical way for businesses to find, customize and create mobile forms to replace paper forms." The company announced new integration with cloud-based storage services to automatically transfer data collected by its mobile apps to Dropbox, Box, Evernote or Google Drive.

For actually creating or editing the custom apps, Canvas supplies an app builder featuring a dashboard to add or manage users, track submissions, view reports and usage.

To get around that pesky coding thing, the company runs an app store with more than 1,400 prebuilt offerings that can be customized for your business. The apps offer incredible specificity, ranging from a waiver and release form to protect pool owners to an app to report injury accidents at laser tag entertainment centers.

If those aren't specific enough, Canvas will convert your first form to an app for free with a trial subscription.

Also getting in on the free app creation market is BuildFire, which last week announced its own mobile app creation platform. "At the heart of the platform is BuildFire's revolutionary 'click-and-edit' dashboard, which provides a simple user interface with no coding required," the company said. If that's too much for you, the company will build your app for free.

"After simply typing in your Web address, BuildFire will deliver a custom, handcrafted, and ready-to-publish native smartphone, tablet, and web app within two business days for free," the company said. The service costs $49 per month after a 14-day trial.

As stated in an introductory video, the service targets organizations that see their customers going mobile and don't want to get left behind, but who have "a limited budget and no real technical skills."

BuildFire also uses templates, such as for small businesses, education, nonprofits and more. "Literally anyone can both build and maintain high-quality and highly engaging apps with the utmost ease," the company said. "Our system is easy to navigate and requires no coding whatsoever. So whether you're a DIYer or a white labeler, we make it easy for you to look like a pro."

Well, I'm a DIYer hobbyist, but I haven't tried any of the above products, so I can't speak to the vendors' claims.

However, I recently decided to try my hand at mobile app building. Having had some previous experience with Java, I targeted the Android platform. It took me the better part of a weekend just to download, install and configure the regular Eclipse IDE and the Google-customized ADT Eclipse IDE and build a simple "Hello World" app and transfer it to my Android device. Granted, a flawed tutorial caused much of that pain. A time-consuming search for a solution revealed the tutorial has been causing the same pain to developers for years but hasn't been corrected, though someone finally provided a missing step, to the thankful accolades of frustrated newbies everywhere.

Anyway, by the time I created anything useful, the industry would have long since moved on. Apps will be beamed to our brains and we'll control them with thoughts. "A mobile device app? What's that?"

Hmm, maybe there's something to this codeless thing after all.

What's your experience with codeless, drag-and-drop tools? Do you and your coding expertise feel threatened? Please comment here or drop me a line.

Posted by David Ramel on August 12, 20140 comments

Doubts About Cross-Platform Mobile Development

It's like a week doesn't go by without multiple PR pings to my inbox announcing the latest cross-platform mobile app development tool release.

Everybody is getting in on the act, even companies you wouldn't expect, like Facebook, Oracle, IBM, Amazon, Intel, Cisco -- even Microsoft. I alone have recently reported on tools like Xamarin, RhoMobile, Appcelerator, Appmethod, AppGyver, PhoneGap, RAD Studio, Icenium and many more.

So cross-platform is the way to go, right? The benefits are well-known: Save time and expense by mostly using one code base to target iOS, Android, Windows Phone and so on. Even traditional concerns about lower performance and less native device functionality were seemingly being addressed by the emergent class of tools for building hybrid apps. Unlike Web-based apps just using HTML5, CSS and JavaScript, the hybrid tools use different types of wrappers -- or containers -- to access native resources such as cameras and accelerometers, usually through JavaScript API calls. The vendors claim this method provides native performance -- or nearly so. It also gets you into the app stores -- unlike Web apps -- and reportedly provides other benefits.

The message is that only under special circumstances would you want to spend your time and effort on multiple projects for platform-specific native development.

Maybe not so much.

"Developers are still struggling with the limitations of cross-platform development," I was surprised to learn in a recent report from Research and Markets, a "market research store." The firm's main premise is that growth of the cross-platform mobile development tools market is slowing dramatically because of a variety of factors. One of those factors is "the increased popularity of native apps" because of the aforementioned struggles with cross-platform tool limitations.

"The performance of Web technologies and access to device-level functionality are cited by developers as significant challenges, pushing them toward native app development," the report stated.

Another factor is "diminishing productivity returns." That's because "developers are reporting less efficiency gains when using cross-platform mobile development tools compared to previous years," the report stated. "The drop is eroding the perceived value of tools."

Well, you know as well as I do to add a grain of salt to any ingestion of hype from research vendors hawking their for-sale reports. The report even contains contradictory statements like, "A shortage of native developers within the enterprise is continuing to help drive growth for cross-platform mobile development tools." Huh?

But that report isn't all that caught my attention.

Another recent report -- this one from VisionMobile Ltd. -- indicates that non-game developers target an average of 1.75 platforms. It seems this number would be higher if cross-platform development was really the craze I thought it was. The research also found that cross-platform tools was listed as the fourth-most popular category of third-party tools, behind ad networks, cloud computing and push notifications.

Furthermore, VisionMobile said, "It might seem counter-intuitive, but there are 26 percent of CPT [cross-platform tools] users who only target a single platform. They are most likely using the CPT to avoid learning the native platform rather than for greater reach or cost efficiency; a decision to help the developer rather than the customer."

The company's "Developer Economics Q3 2014: State of the Developer Nation" report quoted Conny Svensson, an exec at CGI Sweden, as providing one possible explanation for these findings.

"The gap between HTML and native is widening," Svensson said. "The native SDKs introduce new APIs at a faster rate than HTML5 can keep up with. To harness the power of these new APIs, native is the only viable option."

In a section titled, "To Go Native Or Not To Go Native, That Is The Question," the report indicated that among primary platforms, Android is outpacing iOS globally, garnering loyalty by 42 percent of developer respondents as opposed to 32 percent going with iOS. VisionMobile noted that Apple introduced the Swift programming language to make native iOS development more accessible (it also subsequently signed on with IBM).

"Google's strategy, on the other hand, would really like everyone using Web technology so they can more effectively track users for ad targeting purposes," VisionMobile added. "Their latest design and technology moves blur the lines between native apps and the Web."

Further adding to my FUD about cross-platform development is yet another brand-new report, this one by the appFigures "reporting platform for mobile app developers."

"Most App Developers Stick With One Store," an appFigures blog post recently reported. It announced that there were about 272,000 developers targeting the iOS App Store and some 293,000 developers targeting the Android-centric Google Play store. The overlap among those audiences amounted to about 58,000 developers, or roughly 11 percent of the total group. That provided "a strong indication that developers prefer to stick with one store," the report said.

So what's the real story? Is cross-platform mobile app development taking the world by storm or not?

I found it curious that the first two reports I quoted noted the advantages of native development over Web-based technologies, but didn't take into account the burgeoning market of tools to build hybrid apps, which are supposed to close the performance and native functionality gaps.

Brandon Satrom, for one, claims this is possible in an article the Telerik exec penned for VentureBeat titled, "How To Build A Hybrid App That Performs Like Native -- Yes, It Can Be Done."

I'm intrigued by the hybrid approach and the different means that vendors use to supposedly match native performance and functionality, and I've been researching the issue. But I've found little in the way of hard facts, metrics or benchmarks. There are a lot of performance claims by vendors but little supporting evidence.

For example, yet another new report, this one by research2guidance, stated that, "For the vast majority of app developers, the use of CP [cross-platform] tools does not come at the cost of app quality." It went on to say, "81 percent claim that the quality of the apps developed with CP tools is as good as or even better than apps developed with a native IDE."

So please clue me in. If you've read this far, you must be a mobile app developer (or yet another PR person right now typing my e-mail address into your database). What's your experience? Are you going cross-platform or not? Do hybrid apps really match native apps? Comment here or drop me a line. Let's get to the bottom of this.

Posted by David Ramel on August 1, 20140 comments

Programmers Make Big Bucks on Wall Street, but at What Cost?

When we last left our friend Sergey Aleynikov, the Wall Street programmer had been convicted of stealing trade secrets from Goldman Sachs and sentenced to prison in 2010.

Aleynikov was involved in the ultra-secret world of high-frequency trading (HFT), where programmers build the lowest-latency systems in the world to execute market trades milliseconds before competitors, making untold millions of dollars with no human intervention whatsoever. He was convicted of stealing HFT code en route to a new job at a rival trading house.

I came across the case while examining "What Do the Highest-Paid Programmers Make?" It was revealed during the trial that Aleynikov was paid $400,000 per year by Goldman Sachs Group Inc., and the competitor, Teza Technologies LLC, was going to triple that salary. Getting $1.2 million per year to type computer code shows just how high the stakes are in the HFT game -- a game none of the players want to discuss at all. The courtroom was even closed so as not to divulge trade secrets during testimony.

Turns out, Aleynikov's conviction was overturned in appeals court in 2012 after he served about a year in prison.

Turns out, he was re-arrested later that year, and -- according to Wikipedia -- the government changed the laws to ensure what Aleynikov did was more of a convictable offense, as spelled out in the "Theft of Trade Secrets Clarification Act of 2012."

All of which serves to point out a disturbing trend of the government cracking down on programmers who take their employers' code or intellectual property with them when they change jobs, as reported by Bloomberg last week in an extensive article titled "Wall Street Techs Take Secrets to Next Job at Their Peril."

Used to be, such disputes were settled in negotiations or even civil suits. Now, you can go to prison. Which might happen to Kang Gao, recently arrested while leaving a $21 billion quantitative hedge fund to join a competitor. Sound familiar? Bloomberg reported:

Gao is the fourth Wall Street analyst or programmer to be ensnared in a crackdown by the D.A., Cyrus Vance Jr., on intellectual property theft from financial firms. His decision to charge Gao, former Goldman Sachs Group Inc. programmer Sergey Aleynikov and ex-employees of the Dutch firm Flow Traders BV, signals a willingness to criminalize financial industry disputes formerly relegated to contract litigation.

So, yeah, go for the big bucks on Wall Street. Just weigh the price you might have to pay if you try to cross one of those firms and leverage your insider knowledge with another employer. These big boys supply a nice gravy train to the "one-percenters" and they don't want to share the multi-billion-dollar flow with anyone. More from Bloomberg:

The prosecutions come as Wall Street is increasingly protective of intellectual property, including trading models and software code, which have become more valuable as firms seek a millisecond advantage over rivals through strategies including high-frequency trading.

Sure, I understand that the code you create belongs to your employer, not you, and employers have a right to protect their intellectual property. But I'm wondering just what that implies, nowadays. Of course taking full programs or even lines of code with you isn't legal and should be punishable.

But it seems to me this crackdown could be the crest of a slippery slope. With the aggressive new stance on the part of Wall Street firms and the government, will the line blur about what constitutes intellectual property? Will you be allowed to jump ship and take your knowledge with you? If you thought of a great idea behind a successful algorithm, will you be unable to duplicate your work from memory at a new firm? Will you be able to create brand-new programs that work similarly to those at a prior employer? Will you be able to use new esoteric skills you learned on a previous job? Just where will the line be drawn?

The Aleynikov case might supply some clarification. He's still awaiting trial -- or rather, retrial -- and just last week The New York Times reported a judge threw out a bunch of evidence in the case, throwing more uncertainty into the whole bizarre affair.

Meanwhile, the murky world of HFT has come under great scrutiny since I wrote about it more than three years ago.

The Aleynikov case so piqued the interest of Michael Lewis -- he of "Moneyball" sports Big Data fame -- that he wrote about it for Vanity Fair last year in an article titled "Did Goldman Sachs Overstep in Criminally Charging Its Ex-Programmer?" and this year parleyed his research into a new book, "Flash Boys." Elsewhere, it's all over the news, with ongoing debates about whether it's an unfair advantage of the rich or a necessary tool to keep financial institutions alive. Just last week, Reuters reported on a Securities and Exchange Commission investigation into 10 HFT firms.

Regardless of the socio/political debate, the technology is fascinating, as I recounted in a follow-up article, "When Milliseconds Make Millions: Why Wall Street Programmers Earn the Big Bucks."

Aleynikov was a C++ guy, and I figured the HFT gurus were using arcane super-sophisticated technology and techniques that weren't within the reach of most developers, but that turned out not to be the case, at least as far as sophistication goes.

I was surprised last year to see an update to a post about "Slang, Goldman Sachs' proprietary programming language," on Stack Overflow:

Apparently, Goldman Sachs' code for program trading is written in a proprietary programming language called slang and relies on a proprietary database (secdb).

A reader supplied more details, "At the risk of getting sued, let me throw you geeks a bone and part the Goldman veil a bit." The reader used to work at the firm and in his post said "you have no idea how primitive the development culture is on the Street."

On Hacker News, Slang was also under discussion a while back:

I've looked at Slang a bit. It's an interpreted dataflow language running on an in-memory database called SecDB. Untyped, Pascal-ish, single-threaded, and poor support for namespaces. Like a spreadsheet, it only needs to recompute the subgraph that has changed. It was probably innovative 25 years ago (I think it came out mid-80s), but today you could write Java code and use memorization aggressively to speed things up. Slang is not why GS traders are successful.

So, apparently, if Slang isn't the answer to the speed question, Goldman Sachs and friends have been doing a pretty good job of holding on to their secrets. As the firm responded to Lewis' Vanity Fair article:

Goldman Sachs has spent millions of dollars and tens of thousands of hours developing the proprietary source code and technology used in our market-making business.

Rather than having a speed advantage, it's more likely that today's technologies have so evened the latency field that the most important intellectual property is probably the logic behind the trading algorithms: exactly what to do when whatever happens. (Though a German article seems to be saying speed traders are turning to microwave transmissions because they're faster than fiber-optic lines.) Nevertheless, for the tech-heads, QuantStart has a detailed examination of the "Best Programming Language for Algorithmic Trading Systems."

Anyway, I'm not too young to change careers; if you have any insights into the fascinating technology backing HFT, please comment here or drop me a line.

Just hope the FBI isn't listening in.

Posted by David Ramel on July 25, 20140 comments

Mobile, Big Data Development Moves to the Cloud

While not a new trend, the migration of mobile and Big Data development to the cloud has shifted into overdrive lately.

Witness a new report from ABI Research predicting that the growth in cloud application development and management platforms will drive $3.6 billion in mobile enterprise application revenues by 2019.

The research firm said more companies are looking to mobile application platforms -- deployed either on-premises or in the cloud -- for the development and management of their enterprise apps. Cloud-based solutions, however, are expected to outpace their on-premises counterparts, to the tune of 42.5 percent compound annual growth rate (CAGR) from this year to 2019.

"Mobile applications allow for greater flexibility when mobilizing content and employees," said ABI Research senior analyst Jason McNicol. "However, the cost in terms of time and resources is fairly expensive to generate a single app for multiple platforms like iOS, Android and Windows Phone. Fortunately, new cloud-based development solutions have evolved and are now gaining traction to reduce the app development time while permitting cross-platform deployment."

These cloud-based development solutions are divided into two camps, the company said: front-facing and back-end integration.

An example of the back-end trend -- or Mobile Backend as a Service (MBaaS), because everything has to be an "X as a Service" these days -- was just revealed yesterday. Amazon Web Services announced a new mobile SDK and associated services to simplify some of the back-end plumbing like identity and data management, hooking up to storage, sending push notifications, analyzing user behavior, and so on. Think Microsoft Azure Mobile Services or the similar Google initiative just announced two weeks ago.

On the front-end, as ABI Research noted, it's pretty much all about cross-platform app development, an exploding market as evidenced from recent tools announced by Oracle, Embarcadero Technologies, Xamarin, AppGyver, Microsoft and many others.

Meanwhile, in the Big Data arena, things are happening fast as the trend to move large-scale analytics to the cloud accelerates.

The advantage of running your Big Data analytics in the cloud rather than on-premises -- especially for smaller companies with constrained resources -- are numerous and well-known. Oracle summed up some of the major business drivers in the article, "Trends in Cloud Computing: Big Data's New Home":

• Cost reduction
• Reduced overhead
• Rapid provisioning/time to market
• Flexibility/scalability

"Cloud computing provides enterprises cost-effective, flexible access to Big Data's enormous magnitudes of information," Oracle stated. "Big Data on the cloud generates vast amounts of on-demand computing resources that comprehend best-practice analytics. Both technologies will continue to evolve and congregate in the future."

In fact, they will evolve and congregate to form a $69 billion private cloud storage market by 2018, predicted Technology Business Research. That's why the Big Data migration to the cloud is picking up pace recently -- everybody wants a piece of the multi-billion-dollar pie.

As Infochips forecasted early last year: "Cloud will become a large part of Big Data deployment -- established by a new cloud ecosystem."

The following moves by industry heavyweights in just the past few weeks show how that ecosystem is shaping up:

• IBM last week added a new Big Data service, IBM Navigator on Cloud, to its IBM Cloud marketplace. With a reported 2.5 billion gigabytes of data being generated every day, IBM said the new Big Data service will help organizations more easily secure, access and manage data content from anywhere and on any device.

  "Using this new service will allow knowledge workers to do their jobs more effectively and collaboratively by synchronizing and making the content they need available on any browser, desktop and mobile device they use every day, and to apply it in the context of key business processes," the company said.

  The new service joined other recent Big Data initiatives by IBM, such as IBM Concert, which offers mobile, cloud-based, Big Data analytics.

• Google last month announced tools to "help developers build and optimize data pipelines, create mobile applications, and debug, trace, and monitor their cloud applications in production." One such tool added to the company's Cloud Platform suite is Cloud Monitoring, designed to "let developers understand, diagnose and improve systems in production."

  Another is Google Cloud Dataflow, "a fully managed service for creating data pipelines that ingest, transform and analyze data in both batch and streaming modes."

  Dataflow is a successor to MapReduce, a programming paradigm and associated implementation created by Google that was a core component of the original Hadoop ecosystem that was limited to batch processing and came under increasing criticism as Big Data tools became more sophisticated.

  "Cloud Dataflow makes it easy for you to get actionable insights from your data while lowering operational costs without the hassles of deploying, maintaining or scaling infrastructure," Google said. "You can use Cloud Dataflow for use cases like ETL, batch data processing and streaming analytics, and it will automatically optimize, deploy and manage the code and resources required."

• EMC on Tuesday acquired TwinStrata, a Big Data cloud storage company. The acquisition gives traditional storage vendor EMC access to TwinStrata's CloudArray cloud-integrated storage technology.

  That was just one of a recent spate of moves to help EMC remain competitive in the new world of cloud-based Big Data. For example, when the company announced an upgrade of its VMAX suite of data storage products for big companies, The Wall Street Journal reported: "Facing Pressure from Cloud, EMC Turns Data Storage into Service."

  The same day, EMC announced "a major upgrade to EMC Isilon OneFS, new Isilon platforms and new solutions that reinforce the industry's first enterprise-grade, scale-out Data Lake." But wait, there's more: EMC also yesterday revealed "significant new product releases across its Flash, enterprise storage and Scale-Out NAS portfolios" to help organizations "accelerate their journey to the hybrid cloud."

EMC's plethora of Big Data/cloud announcements make it clear where the company is placing its bets. As financial site Seeking Alpha reported: "EMC Corporation: Big Data and Cloud Computing Are the Future."

That was in March, and the future is now.

Posted by David Ramel on July 11, 20140 comments

Interactive IEEE Tool Ranks Programming Language Popularity

Nothing gets developers going like a discussion of programming languages. I still have to stop and click on each new article purporting to tell us which is the best or most popular language. (Part of the attraction is wading through the inevitable flame wars that break out in the reader comments section. Things are always guaranteed to get lively.)

There's never a "right" answer, of course, so a new tool published by IEEE Spectrum lets you rank languages according to your own criteria (the tool's sources and methodology are explained here).

IEEE Spectrum, the flagship Web site and publication of the IEEE (formerly the Institute of Electrical and Electronics Engineers), provides the tool with several customization options.

You can limit the rankings to certain sectors or combinations of sectors: Web, mobile, enterprise and embedded. You can view the IEEE Spectrum "overall" ranking or choose trending (growing rapidly); jobs (in demand by employers); open (popular on social media and open source hubs); or custom (design your own ranking).

The rankings use 12 weighted data sources, and the custom weighting option lets you choose which sources to use and how much import to assign to them with sliders. Sources include Google search and trends; GitHub active and created; Stack Overflow questions and views; Reddit; Hacker News, career sites; and several more.

"We separated some sources, such as Google, into two metrics, one of which is more sensitive to queries made by people asking questions, while the other is more sensitive to resources used by people with a fairly good grasp already," said co-author Stephen Cass in reply to a reader comment about the methodology behind the tool. "But unfortunately, every metric is by necessity a proxy of some sort, subject to these kinds of effects, since we can't monitor language use directly."

Unsurprisingly, no matter how you slice the criteria, it's pretty much Java right on down the line.

The overall top 10 with all sectors in play reads:

1. Java
2. C
3. C++
4. Python
5. C#
6. PHP
7. JavaScript
8. Ruby
9. R
10. MATLAB

The topmost languages are mostly the same for the different sectors, though "embedded" shakes things up, putting C as No. 1 and adding Assembly and Arduino to the top 10.

The IEEE tool results are basically in line with other such measurements.

For example, here's the latest TIOBE index for July:

1. C
2. Java
3. Objective-C
4. C++
5. (Visual) Basic
6. C#
7. PHP
8. Python
9. JavaScript
10. Transact-SQL

Here's the latest from the LangPop.com site, which features Programming Language Popularity charts (not updated since last October) that also use multiple sources and allow for user weighting:

1. C
2. Java
3. PHP
4. JavaScript
5. C++
6. Python
7. Shell
8. Ruby
9. Objective-C
10. C#

Here's the latest from the PYPL (PopularitY of Programming Language) index:

1. Java
2. PHP
3. Python
4. C#
5. C
6. C++
7. JavaScript
8. Objective-C
9. Ruby
10. Swift

The PYPL is based on Google Trends data. So going back to the IEEE tool and choosing Google Trends as the only criterium should produce the same results:

1. C
2. Java
3. C++
4. Python
5. C#
6. PHP
7. JavaScript
8. R
9. Visual Basic
10. SQL

It doesn't, probably because the two tools use slightly different methodology. For example, one big difference is that the IEEE tool searched for "X programming" in January 2014, while the PYPL tool also used the word "tutorial."

All such tools take different approaches, and thus can't be compared and contrasted. And there are always anomalies. For example, the IEEE tool seems to place R way up there, for some reason. And I find it hard to believe Swift already has gained top-10 traction as reported by PYPL.

So the usefulness of such tools is questionable, but they're sure fun to play around with and pass the time when you should be programming.

And don't forget, you can read the user comments for interesting debates such as whether or not Java is an embedded language, or if it should be PERL (as an acronym for Practical Extraction and Report Language) or just Perl.

Posted by David Ramel on July 7, 20140 comments

Agile & Pair Programming: Scary but Necessary

Is it really a good idea to team up two developers to work on the same piece of source code, actually sitting together and watching one another type?

Programmers are known by some for being introverted loners, sometimes lacking social graces. They're known by some as artists, extremely touchy about their code and style. They're known by some for keeping strange hours, sometimes churning out their best artistic creations in long, caffeine-fueled, late-night coding jags with who-knows-what screaming through their headphones. They're known by some as -- ahem -- practicing questionable hygiene.

And you're going to tell them that the company is going Agile and adopting one of the major tenets of that methodology, pair programming -- and the fat older guy with the ponytail is going to be married to the young skinhead dude representing with all the tats for the duration of an important project piece?

Really?

"It is scary," admitted Dr. Mark Balbes, vice president of Architecture at Asynchrony, during a new webcast titled "Stories, Pair Programming & More: Top Must-Have Agile Techniques/Best Practices," hosted by ADTmag.com editor-at-large John K. Waters.

"It's scary because there's somebody that's right there who's going to be looking at your code while you're thinking," continued Balbes, author of the popular The Agile Architect column on this site. "And you don't have the opportunity to sit in a cube somewhere and tap out your code and refactor it and rewrite it and only when you're done -- and you're proud of it -- handing it off to somebody else for a code check. That pair is sitting right there watching you type every character. And it takes time to get used to that."

The key to acceptance is conveying to the team the exciting opportunity the methodology presents, Balbes said. The Agile coach needs to explain to the teams how fast everyone is going to be learning from everyone else. Knowledge of a tool or technology transfers to others much more quickly in a pair environment, he said.

Fellow panel member Jason Tice, director of Agile Coaching and Training Practice at Asynchrony, agreed that coaching is key. "Acknowledge that pairing is something that needs to be coached, Tice said. "Agile is a social process, it is not a technical process. And some developers are of the minds that they don't like to communicate that much."

Tice said he has coached teams to overcome such challenges, but it's sometimes tough to do. He emphasized that team members need to be accepting and accommodating of each other, adding that he coaches teams to talk about such issues out in the open and discuss how they can make the process comfortable for everyone.

"Sometimes you have people that have hygiene issues," Tice said. "They have physical space issues. They don't like someone being right next to them. They might have some issues that they're not confident in the code base and they're afraid someone's going to point the finger at them and say they don't have the technical skills to be on the team."

Again, the team leader needs to explain that pair programming is what they're going to be doing in order to write better-quality software, and if there are issues, the team should talk about it and figure out how to be successful, he said.

Matt Philip, the third member of the webcast panel, agreed that coaching is key, but warned that encouraging developers to try the practice is better than forcing it on them, if possible.

"Any time you kind of impose practices or impose a process, it gives people a way out. They don't have to own the decision for themselves," said Philip, the learning and development architect at ThoughtWorks Inc. Also, he said, it's best to find developers whose personalities better suit pair programming, if possible. At ThoughtWorks, he said, recruiters try to make sure candidates are comfortable with working alongside people and pairing up with others.

The IBM-sponsored webcast is available for replay for those who want to learn more about Agile, including how to sell it to management, essential practices to follow, how to start learning about the methodology, how to make it work with distributed teams in different countries and time zones, how to pronounce Kanban, and much more.

Posted by David Ramel on May 27, 20140 comments

Intel on Android vs. Windows Tablet Development: We Go Where the Market Is

Remember Wintel? Since the birth of the original PC, Windows and Intel have been aligned in a marriage of cooperation and profitability.

Not so much anymore.

With traditional PCs dying and mobile devices such as tablets thriving, Intel is increasingly moving to the dominant tablet OS, Android. That's not news, of course, but the trend has become even more prominent recently, and Intel is actively courting Android developers with a new cross-platform development tool, still in a beta preview.

In a conference call held this week to discuss company first-quarter financials, Intel executives disclosed they sold 5 million tablet processors so far this year -- mostly for Android -- and expect that number to grow to 40 million at year's end.

When an analyst asked Intel CEO Brian Krzanich about what percentage of those chips will be running on Android, he answered:

Our mix of OSes reflects pretty much what you see in the marketplace. It's probably something on the order of 90 percent Android, 80 percent Android, 10 to 20 percent Windows. Our percentages look very much like the marketplace. So if Windows continues to grow and gain traction, I think our percentage would just align directly to that. Don't separate what we ship from what's basically in the marketplace.

Krzanich also touted the cross-platform capabilities of Intel's architecture. "One of the things an OEM gets when they build with Intel technology is that they can go into any OS and they can build a single platform and move that onto Chrome, onto Android, onto Windows, and that's a very unique capability that we provide to OEMs for flexibility," he said.

Even though the tablet business isn't yet as profitable as the PC business in terms of revenue and margins, Krzanich said it's "critical" to Intel for two reasons, one being scale and the need to be involved in all areas of personal computing.

"The second one is developer attention," Krzanich said. "You want developers creating new products, doing innovation on your architecture. This is a space that's got innovation. We are going to bring some of that innovation to this market."

That innovation, he said, "keeps the developers developing and interested in our platform. I think for all of those reasons, we want to be in this space and we will be in this space from now on."

Which brings us back to that development tool. The Integrated Native Development Experience (INDE) is a beta release of a cross-platform development suite, designed "to quickly and easily create applications targeting Android and Windows devices with native performance, outstanding battery life, and exposure to unique platform capabilities."

Ironically, it just runs on 64-bit Windows, but under "OS Target Support," Android is listed first, ahead of Windows. Mac OS X and Linux support are coming.

The tool supports development in C++ and Java and works with the Eclipse and Microsoft Visual Studio IDEs, the latter with the vs-android plug-in.

It features special components for handling media, threading and executing code on devices beyond the CPU.

The INDE development suite "enables efficient reuse of performance-sensitive code across operating systems and platforms with consistent C++/Java tools, libraries and samples for environment setup, code creation, compilation, debugging and analysism," Intel said in a product brief.

Developers wanting to take part in the beta can visit the "Intel INDE Opt-in Free Trial" page, which states: "To encourage developers to build Android applications for Intel architecture-based platforms, we are discounting [INDE] and the tools it contains for Android to $0 for a limited-time, 12-month license."

Will Microsoft catch up in the mobile OS marketplace? Please comment here or drop me a line.

Posted by David Ramel on April 18, 20140 comments