Computed·Blg

Via Recursivity.blog()

-----

At the beginning of last week, I launched GreedAndFearIndex - a SaaS platform that automatically reads thousands of financial news articles daily to deduce what companies are in the news and whether financial sentiment is positive or negative.

It’s an app built largely on Scala, with MongoDB and Akka playing prominent roles to be able to deal with the massive amounts of data on a relatively small and cheap amount of hardware.

The app itself took about 4-5 weeks to build, although the underlying technology in terms of web crawling, data cleansing/normalization, text mining, sentiment analysis, name recognition, language grammar comprehension such as subject-action-object resolution and the underlying “God”-algorithm that underpins it all took considerably longer to get right.

Doing it all was not only lots of late nights of coding, but also reading more academic papers than I ever did at university, not only on machine learning but also on neuroscience and research on the human neocortex.

What I am getting at is that financial news and sentiment analysis might be a good showcase and the beginning, but it is only part of a bigger picture and problem to solve.

Unlocking True Machine Intelligence & Predictive Power
The human brain is an amazing pattern matching & prediction machine - in terms of being able to pull together, associate, correlate and understand causation between disparate, seemingly unrelated strands of information it is unsurpassed in nature and also makes much of what has passed for “Artificial Intelligence” look like a joke.

However, the human brain is also severely limited: it is slow, it’s immediate memory is small, we can famously only keep track of 7 (+-) things at any one time unless we put considerable effort into it. We are awash in amounts of data, information and noise that our brain is evolutionary not yet adapted to deal with.

So the bigger picture of what I’m working on is not a SaaS sentiment analysis tool, it is the first step of a bigger picture (which admittedly, I may not solve, or not solve in my lifetime):

What if we could make machines match our own ability to find patterns based on seemingly unrelated data, but far quicker and with far more than 5-9 pieces of information at a time?

What if we could accurately predict the movements of financial markets, the best price point for a product, the likelihood of natural disasters, the spreading patterns of infectious diseases or even unlock the secrets of solving disease and aging themselves?

The Enablers
I see a number of enablers that are making this future a real possibility within my lifetime:

• Advances in neuroscience: our understanding of the human brain is getting better year by year, the fact that we can now look inside the brain on a very small scale and that we are starting to build a basic understanding of the neocortex will be the key to the future of machine learning. Computer Science and Neuroscience must intermingle to a higher degree to further both fields.
• Cloud Computing, parallelism & increased computing power: Computing power is cheaper than ever with the cloud, the software to take advantage of multi-core computers is finally starting to arrive and Moore’s law is still advancing at ever (the latest generation of MacBook Pro’s have roughly 2.5 times the performance of my barely 2 year old MBP).
• “Big Data”:  we have the data needed to both train and apply the next generation of machine learning algorithms on abundantly available to us. It is no longer locked away in the silos of corporations or the pages of paper archives, it’s available and accessible to anyone online.
• Crowdsourcing: There are two things that are very time intensive when working with machine learning - training the algorithms, and once in production, providing them with feedback (“on the job training”) to continually improve and correct. The internet and crowdsourcing lowers the barriers immensely. Digg, Reddit, Tweetmeme, DZone are all early examples of simplistic crowdsourcing with little learning, but where participants have a personal interest in participating in the crowdsourcing. Combine that with machine learning and you have a very powerful tool at your disposal.

Babysteps & The Perfect Storms
All things considered, I think we are getting closer to the perfect storm of taking machine intelligence out of the dark ages where they have lingered far too long and quite literally into a brave new world where one day we may struggle to distinguish machine from man and artificial intelligence from biological intelligence.

It will be a road fraught with setbacks, trial and error where the errors will seem insurmountable, but we’ll eventually get there one babystep at a time.
I’m betting on it and the first natural step is predictive analytics & adaptive systems able to automatically detect and solve problems within well-defined domains.

Via giagom

-----

Can ARM wrestle its way into the server market? Calxeda and Hewlett-Packard think so. On Tuesday Calxeda launched its EnergyCore ARM server-on-a-chip (SoC), which it says consumes as little as 1.5 watts (and idles at half a watt). And HP, the world’s largest server maker, committed to building EnergyCore-based servers that will consume as little as 5 watts when running all out. Compare that to the lowest-power x86 server chips from Intel, which consume about 20 watts but deliver higher performance.

Calxeda, backed in part by ARM Holdings, is banking that the success that ARM chips found in smartphones and mobile devices will carry over into data centers serving large, scale-out workloads. In that arena, it is facing off squarely against chip giant Intel and its x86-based architecture, which dominates the market for chips running in commodity servers.

Said Calxeda in a statement:

Thanks to its mobile phone heritage and patent-pending innovations from Calxeda, the new processor consumes less than one tenth the power of today’s most energy-efficient server processors and is ideal for workloads such as web serving, ‘Big Data’ applications, scalable analytics such as Apache Hadoop, media streaming and mid-tier infrastructure such as caching and in-memory databases.

EnergyCore targets web serving, big data apps

The small form factor and energy stinginess of EnergyCore, based on the ARM Cortex processor, suits an emerging and fast-growing class of web and cloud applications, but it lacks in terms of software support and currently won’t support the enterprise demand for 64-bit processors. Thus, for traditional data centers locked into the Intel x86 architecture and with lots of legacy software to run, Calxeda might be a stretch. But that might not matter.

“For big cloud companies that buy gargantuan numbers of servers and for whom the power and space issues get linearly nasty as they build up the number of nodes, this is a good solution,” said analyst Roger Kay, the founder and president of Endpoint Technologies Associates.

These sorts of transactions take on an almost consultative nature, where the server vendor works with the customer’s developers, he said.

EnergyCore is 32-bit only, a fact that Intel will no doubt trumpet. “High-performance computing [HPC] needs 64 bits to deal with larger address space, but that doesn’t mean that 32-bit [processors] can’t address certain data center applications,” Kay said. “This new chip is designed to handle very large databases that in turn handle lots of queries from many end points. Think Google Earth where there are lots of simple queries — ‘show me the bits in the X-Y grid specified.’”

HP estimates that those big scale-out web and cloud data center scenarios represent a healthy 10 to 15 percent of the data center market, Kay noted. That’s certainly worth fighting for. Intel pegs that segment at 10 percent of the overall market.

Richard Fichera, the VP and an analyst at Forrester Research, said Calxeda did its homework. “This looks to be at least three to five times more energy efficient than other chips and [energy use] is a growing concern for data centers.” Some of what Calxeda has done will be hard for competitors to replicate, he said.

Calxeda, like SeaMicro, which makes low-power servers running Intel Atom processors, also builds on a fabric that lets all the various SoC components communicate inside the box.

Skeptics point out that big data center buyers tend to be a conservative lot, not likely to gamble on a new chip architecture. “Many CIOs will go with the devil they know. They have software that runs on Intel so why move?” Kay said. But again, Calxeda and HP are seeking out the biggest of the big data center companies — those that have a lot of in-house development talent that are not as bound by legacy software concerns.

What’s in EnergyCore SoC?

• Multicore ARM Cortex processor running at 1.1 GHz to 1.4 GHz
• Supports FPU (scalar) and NEON (SIMD) Floating Point
• 4 MB of onboard shared L2 cache
• Integrated Memory Controller with 72-bit datapath, and ECC
• Typical maximum power consumption (running 100% CPU load under normal conditions) is 5 watts
• Typical idle power consumption is less than .5 watts

Calxeda and HP will start rolling out sample products later this year, with volume ramping up in 2012. Calxeda is not alone in this arena: Marvell Inc. is already in the market with its own ARM-based servers. NVIDIA is also building an ARM-based server, dubbed Denver, for HPC.

While Calxeda is tiny compared to Intel, it also doesn’t manufacture its own silicon, which could end up hurting it when comparing it to Intel, one of the last silicon vendors to own its own chip manufacturing. Fichera notes that Calxeda has no control over the distribution and sales of what it designs: The server partner has all the leverage. If someone else has a better SoC next year, Calxeda (or whatever SoC provider they use) could be gone.

Via LUKEW

By Luke Wroblewski

-----

As mobile devices have continued to evolve and spread, so has the process of designing and developing Web sites and services that work across a diverse range of devices. From responsive Web design to future friendly thinking, here's how I've seen things evolve over the past year and a half.

If you haven't been keeping up with all the detailed conversations about multi-device Web design, I hope this overview and set of resources can quickly bring you up to speed. I'm only covering the last 18 months because it has been a very exciting time with lots of new ideas and voices. Prior to these developments, most multi-device Web design problems were solved with device detection and many still are. But the introduction of Responsive Web Design really stirred things up.

Responsive Web Design

Responsive Web Design is a combination of fluid grids and images with media queries to change layout based on the size of a device viewport. It uses feature detection (mostly on the client) to determine available screen capabilities and adapt accordingly. RWD is most useful for layout but some have extended it to interactive elements as well (although this often requires Javascript).

Responsive Web Design allows you to use a single URL structure for a site, thereby removing the need for separate mobile, tablet, desktop, etc. sites.

For a short overview read Ethan Marcotte's original article. For the full story read Ethan Marcotte's book. For a deeper dive into the philosophy behind RWD, read over Jeremy Keith's supporting arguments. To see a lot of responsive layout examples, browse around the mediaqueri.es site.

Challenges

Responsive Web Design isn't a silver bullet for mobile Web experiences. Not only does client-side adaptation require a careful approach, but it can also be difficult to optimize source order, media, third-party widgets, URL structure, and application design within a RWD solution.

Jason Grigsby has written up many of the reasons RWD doesn't instantly provide a mobile solution especially for images. I've documented (with concrete) examples why we opted for separate mobile and desktop templates in my last startup -a technique that's also employed by many Web companies like Facebook, Twitter, Google, etc. In short, separation tends to give greater ability to optimize specifically for mobile.

Mobile First Responsive Design

Mobile First Responsive Design takes Responsive Web Design and flips the process around to address some of the media query challenges outlined above. Instead of starting with a desktop site, you start with the mobile site and then progressively enhance to devices with larger screens.

The Yiibu team was one of the first to apply this approach and wrote about how they did it. Jason Grigsby has put together an overview and analysis of where Mobile First Responsive Design is being applied. Brad Frost has a more high-level write-up of the approach. For a more in-depth technical discussion, check out the thread about mobile-first media queries on the HMTL5 boilerplate project.

Techniques

Many folks are working through the challenges of designing Web sites for multiple devices. This includes detailed overviews of how to set up Mobile First Responsive Design markup, style sheet, and Javascript solutions.

Ethan Marcotte has shared what it takes for teams of developers and designers to collaborate on a responsive workflow based on lessons learned on the Boston Globe redesign. Scott Jehl outlined what Javascript is doing (PDF) behind the scenes of the Globe redesign (hint: a lot!).

Stephanie Rieger assembled a detailed overview (PDF) of a real-world mobile first responsive design solution for hundreds of devices. Stephan Hay put together a pragmatic overview of designing with media queries.

Media adaptation remains a big challenge for cross-device design. In particular, images, videos, data tables, fonts, and many other "widgets" need special care. Jason Grigsby has written up the situation with images and compiled many approaches for making images responsive. A number of solutions have also emerged for handling things like videos and data tables.

Server Side Components

Combining Mobile First Responsive Design with server side component (not full page) optimization is a way to extend client-side only solutions. With this technique, a single set of page templates define an entire Web site for all devices but key components within that site have device-class specific implementations that are rendered server side. Done right, this technique can deliver the best of both worlds without the challenges that can hamper each.

I've put together an overview of how a Responsive Design + Server Side Components structure can work with concrete examples. Bryan Rieger has outlined an extensive set of thoughts on server-side adaption techniques and Lyza Gardner has a complete overview of how all these techniques can work together. After analyzing many client-side solutions to dynamic images, Jason Grigsby outlined why using a server-side solution is probably the most future friendly.

Future Thinking

If all the considerations above seem like a lot to take in to create a Web site, they are. We are in a period of transition and still figuring things out. So expect to be learning and iterating a lot. That's both exciting and daunting.

It also prepares you for what's ahead. We've just begun to see the onset of cheap networked devices of every shape and size. The zombie apocalypse of devices is coming. And while we can't know exactly what the future will bring, we can strive to design and develop in a future-friendly way so we are better prepared for what's next.

Resources

I referenced lots of great multi-device Web design resources above. Here they are in one list. Read them in order and rock the future Web!

• Effective Design for Multiple Screen Sizesby Bryan Rieger
• Responsive Web Design (article) by Ethan Marcotte
• Responsive Web Design (book) by Ethan Marcotte
• There Is No Mobile Web by Jeremy Keith
• mediaqueri.es by various artisits
• CSS Media Query for Mobile is Fool’s Gold by Jason Grigsby
• Why Separate Mobile & Desktop Web Pages? by Luke Wroblewski
• About this site... by Yiibu
• Where are the Mobile First Responsive Web Designs? by Jason Grigsby
• Mobile-First Responsive Web Design by Brad Frost
• Mobile-first Media Queries by various artists
• The Responsive Designer’s Workflow by Ethan Marcotte
• Responsible & Responsive (PDF) by Scott Jehl
• Pragmatic Responsive Design (further details) by Stephanie Rieger
• A Closer Look at Media Queries by Stephen Hay
• Responsive IMGs — Part 1 by Jason Grigsby
• Responsive IMGs — Part 2 by Jason Grigsby
• Device detection as the future friendly img option by Jason Grigsby
• Responsive Video Embeds with FitVids by Dave Rupert
• Responsive Data Tables by Chris Coyier
• RESS: Responsive Design + Server Side Components by Luke Wroblewski
• Adaptation (PDF) by Bryan Rieger
• How I Learned to Stop Worrying and Set my Mobile Web Sites Free by Lyza Danger Gardner
• The Coming Zombie Apocalypse by Scott Jenson
• Future Friendly by various artists