Bruno Borges

We can't all be human encyclopedias, and once in a while, you're bound to stumble upon a conversation on a topic you know absolutely nothing about. To avoid being left out or seeming ignorant, here are a few tips for "tricking" people into thinking you're well informed. More »

Daily deals site Groupon is launching its IPO roadshow this week and is seeking a $10 billion valuation for the company, less than half of what was rumored when the company first filed to go public in June. That’s a big haircut.

With the recent turmoil in the markets, it’s a terrible time to bring a new offering to market. But Groupon may have no choice: based on the numbers we’ve seen from the company to date, it’s burning through cash and may have to go public in order keep the lights on.

According to the company’s latest S-1 filing, its merchant liabilities have increased 19% from $392 to $467 million in the last quarter. Its current liabilities exceed its current assets by more than $300 million.

Groupon uses money from new deals to pay off merchants from previous deals, so Groupon’s continued existence is dependent on consumers and merchants having confidence in Groupon’s continued existence.

In recent weeks, Groupon has experienced a barrage of negative publicity from a wide range of media outlets, including The New York Times. Neither Reuters nor The Associated Press had a positive quote from analysts in their coverage of the company.

If consumers believe that Groupon deals might not be honored and stop buying them, the company will be starved for cash and will collapse.

If merchants believe that they might not be paid for Groupons that they issue and stop running Groupons, the company will be starved for cash and will collapse.

If markets were perfectly efficient, Groupon would collapse overnight like Lehman Brothers.

Groupon bears a lot of similarity to the subprime mortgage crisis:

• No one knows how much value is out there in outstanding Groupons. No one has kept track. Groupon is working to improve this, but many Groupons are still tracked by pen and paper or not tracked at all.
• Groupon issues big checks to merchants without any credit check or due diligence. Groupon is taking on a lot of risk of merchants not honoring their obligations. See my earlier VentureBeat story on the challenges that relying on small businesses poses for Groupon.
• It’s impossible to tell with certainty who will owe money to whom if Groupon fails.
• Banks don’t fully understand the risk they are taking on.
• Groupon merchants don’t fully understand the service that they are buying. Groupon is pitched to merchants as a “no risk” way of reaching consumers. There may be no money down, but running a Groupon has a lot of long-term risk, including losing money on poorly targeted customers and damaging your Yelp ratings.
• Much like housing market models that relied on housing prices to continue to grow, Groupon’s model relies on continued revenue growth. With the latest S-1, we saw that once Groupon slows investment in marketing, revenue growth slows down substantially.

Here’s my best analysis of who stands to lose if Groupon collapses:

Consumers. Groupon purchasers could lose between $500 million and $1 billion. This depends on two major factors: the extent to which merchants choose to honor Groupons and the ability to dispute charges with credit card companies.

Recommendation: Consumers should only purchase Groupons they can use within 60 days. Avoid Groupons for big ticket purchases (like travel) far into the future.

Small businesses. Merchants who sell products and services through Groupon stand to lose up to $500 million. In the United States and Canada, merchants are typically paid within 60 days of the date the Groupon runs. In the rest of the world, merchants are typically paid after the Groupon is redeemed. The big question is what merchants choose to do in the case of a Groupon failure. Do they continue to honor Groupons that were issued? Or do they tell consumers, “Sorry, not our problem?” Continuing to honor Groupons would be the right customer service move but bad for profits. This will vary by merchant.

Recommendation: While I’ve long advised most categories of small businesses to avoid running Groupons just based on the terrible economics, I believe businesses should consider Groupon credit risk. If the cash from running Groupons is critical to your business, I would recommend against running them.

Credit card companies. These are potential losers that may surprise a lot of people. Companies such as Chase Paymentech and American Express could also lose hundreds of millions. The companies essentially serve as a backstop for consumers.

According to Visa’s Ted Carr, for 60 days after the statement on which a charge appears, “Visa operating regulations provide card issuers and cardholders with protection when a merchant ceases operations and the cardholder does not receive what was paid for (the merchant’s bank/acquirer is liable.)”

American Express’s Marina Hoffmann Norville said it’s possible that consumers would be able to dispute charges for longer than 60 days. “We will determine on a case-by-case basis how we will address disputes in those instances where a merchant goes bankrupt or abruptly ceases operation,” she said.

Other credit card issuers like Citi and Bank of America may also lose if they choose to issue refunds in the name of good customer service and can’t recover the money from Groupon’s merchant bank.

With new products like Groupon Getaways, this risk gets even bigger. Instead of a $40 transaction, many transactions are in the hundreds of dollars.

The risk is magnified by the fact that there are often no clear records of Groupon redemptions. Unscrupulous consumers may seek refunds even after they’ve used a Groupon.

Recommendation: Credit card companies are in a tough spot. They can increase the holdback that they have for Groupon. This means that they would hold back more money to cover potential losses. But this would hurt Groupon’s cash flow; that in itself could cause Groupon to collapse. Frontier Airlines was forced into bankruptcy when its credit card processor, First Data, decided to hold back more of the proceeds from ticket sales.

Business partners. Groupon spends hundreds of millions on advertising, much of it on the Internet. Ad networks and publishers stand to lose any unpaid money.

Recommendation: Companies who are heavily dependent on Groupon advertising revenue should look closely at the risk exposure that they have and consider whether the payment terms they give to Groupon are appropriate. This is especially true for ad networks that pay out to publishers in advance of receiving payment.

City of Chicago. Chicago has built its tech reputation on Groupon’s meteoric rise. In many ways, this has been undeserved because Groupon is fundamentally a sales and marketing company, not a technology company. (Fewer than 5% of Groupon’s employees are in technology; many of those are based in Palo Alto.) Some Chicagoans are worried about the impact a Groupon collapse will have on the psyche and reputation of the city.

Rick Summer, a senior equity analyst for Morningstar, a Chicago-based investment research firm, says, “I’m pretty neutral on Groupon’s importance to the city. But I take the longer-term prudent view of’ ‘freeing up good people to run better business.’”

Employees. Groupon employs more than 10,000 people. They stand to lose their jobs and any unpaid wages.

Investors. Of course, public and private investors stand to lose their entire investment. Summer’s analysis values Groupon at $5 billion, less than half the value that the company is seeking. That’s also less than the reported $6 billion that Google offered for Groupon. In Morningstar’s research report, he writes, “IPO investors face a nontrivial risk of permanent capital impairment.” That’s a fancy way of saying IPO investors stand to lose their shirts.

I contacted Groupon to give the company the opportunity to respond to my assertions here, but it has declined to comment (SEC regulations typically prevent pre-IPO companies from making public statements).

My points here are not idle speculation. I believe that without significant business model changes, Groupon could cease to exist in the next 12 to 24 months. The company’s recent product announcements seem to be rehashes of long-established online commerce plays (travel, liquidation of unwanted goods) or products that will have little short-term impact (Groupon Now).

A successful IPO will alleviate some of these issues in that it will give the company more of a cash cushion. But the increased scrutiny that comes with being a public company will bring even more attention to every move the company makes. If that news continues to be negative, it will weaken confidence in Groupon.

This same analysis (except my reference to the City of Chicago, of course) applies to all of the companies in the group buying space, with the exception of Google. (Google’s sizable cash horde makes credit risk and consumer confidence non-issues.)

Last week, we saw BuyWithMe, another group buying company, lay off more than half of its staff after it failed to secure more financing. BuyWithMe didn’t have nearly the scale that Groupon does, but what happens to it could be a good indicator of what will happen with Groupon.

Rocky Agrawal is an analyst focused on the intersection of local, social and mobile. He is a principal analyst at reDesign mobile. Previously, he launched local and mobile products for Microsoft and AOL. He blogs at http://blog.agrawals.org and tweets at @rakeshlobster.

Filed under: deals, VentureBeat

Sometime this week, Google Reader will retire its many confusing sharing features, in attempt to force their small but passionate community to relocate to Google+. It's part of the company's portal-wide redesign effort that will soon affect Blogger and Gmail, where Google will be adding new features instead of taking them away. But if you're a devoted Google Reader user, however, the chore of figuring out how to recreate your old Reader experience in the absence of friending and following far outweighs the upgrades like fancy new button gradients and Google+ integration. Don't worry. We've got you covered.

Creating Reader-Specific Circles

We borrowed the term "Reader-specific circles" from Google's executioner-like blog post about the outgoing social features: friending, following and sharing. This is the full-conversion solution that Google hopes most Reader users decide to pursue. If you're familiar with Google+, setting up the Circles is dead simple, but finding those friends and adding them to the circle one at a time is huge pain. (Unless, as hinted in the official blog post, Google launches an easier export feature with the redesign.) Google+ also won't perfectly re-create the Reader sharing experience. And if you had any doubts about how badly Reader needed a redesign, the number of total steps and illustrations needed to sort everything out will dash them.

If you share items publicly in Reader, you don't have to change a thing - All of your Reader shares should be viewable by everyone following you. You just have to tell them to add you to their Circles in Google+. 

Create a specific circles for your Reader friends

1. Go to your Google Plus homepage and click the Circles icon at the top of the page (or this hyperlink). This will pull up all of your existing Circles and suggest a bunch of friends to add to them.
2. Feel free to start dragging and dropping people into the "Create a New Circle" bubble on the lefthand side, but it'll take you a while if you have a lot of people to add. We've figurd out a work-around so that you can add everyone at once.

Move all of your Reader followers and follows into Circles

1. Click on the "Share Settings" button at on the lefthand side of the page. It should be a link at the bottom of the "People You Follow" box. (You might have to click the + sign to expand the box.) It might be useful to name it "Reader Followers."
2. Click on the link for "### people are following you." This should bring up a lightbox showing all of your followers.
3. Once you see the list of people following you, open a new window with your new Google+ Circle. The only way to make sure your old followers will see your new Google+ posts is to add them one-by-one to your new Reader share circle. (Are you confused yet? It's confusing.)
4. If you want to create a different circle of people that you follow, just re-do step one--name it "Reader Follows" or something. On step two, instead of clicking on the link for "### people are following you" click the link that reads "You are following ### people." This will pull up a list of people you're following that you can then add into your new "Reader Follows" Circle. Now whenever you want to read your old followers shared items, just click the "Reader Follows" Circle on the lefthand side of the Google+ homepage.

Some Alternatives to Google Reader

If you're feeling burned by Google and want to ditch it altogether, there are a couple other RSS readers on the web that we'd recommend. However, social RSS readers have been hit hard by the successes of sharing on Facebook and Twitter so the once crowded marketplace that included start-ups like Streamy and Feed Each Other is now pretty barren. Tumblr also used to support RSS, but they axed the feature this summer. We'll be the first to say that the Reader alternatives are not great, and it might be time to admit it: RSS is dead.

Netvibes - Once a Google Reader competitor, Netvibes is a decent RSS reader with very basic sharing options: email, Facebook and Twitter. There's some bonus functionality in being able to set up Netvibes as an iGoogle-like homepage, if you're into that kind of thing. Bloglines is a very similar service.

BuzzBlaze - Earlier this year, The Next Web posted a preview of BuzzBlaze, an application that looks like a next-generation Reader. However, for now it's in private beta, but you can sign up for early access.

Google+ - As much as it hurts to admit it, Google+ is probably as close as you're going to get. Five months after launch, the social network is facing flagging traffic, struggling to keep people on the site and not making much progress with power users, like celebrities (as Twitter did) and brands (as did Facebook). But Google+ isn't finished yet. In the past few days, a small wave of new features have been splashing around in other Google products, like Reader and Blogger, and Google executives have announced that they'll soon allow pseudonyms on the site. It might even get a music store!

Last week, Google announced an impending update to its neglected online RSS reader, Google Reader. The service is soon getting a cosmetic overhaul and will see its current social features removed in favor of deeper integration with Google+. Although many TechCrunch commenters lamented alongside me about the forthcoming disappearance of our favorite “human curators,” when I asked the same question on Google+, the outcome was decidedly different: no one really cares.

Except, it appears, Iranians. According to a blog post now making its way around the Web, Google Reader served the Iranian community as a way to get uncensored, unfiltered news outside of government control. And now, that may be over.

As explained by Amir on Amirhm.com, Google Reader is not a separate domain (i.e., it’s available at www.google.com/reader) and it’s available behind a secure URL beginning https. This setup makes it hard for the government to directly block and filter Reader, even though many other social services, including Twitter, Facebook, FriendFeed, YouTube, Tumblr, Flickr and Picasa, are routinely banned in Iran, a country that’s ranked as the world’s worst oppressor of online freedoms.

In Iran, Reader is able to serve as a hidden social outlet. It even has super-users like activist VahidOnline, a user with more than 7500 followers. These folks help share and spread news through Reader with posts that become online discussion boards for a network of Iranian users.

Even though my simple question about Google Reader’s soon-to-be-removed social features got a big “who cares?” on Google+, if you look in the right places, you’ll see Iranians begging Google to reconsider (oddly, also on Google+, which has apparently not yet been shut down there.)

Case in point.

And again. And again. And again. And again. And again.

Some users are even going so far as to accuse Google of cooperating with Iranian governmental censors, given this move. That’s not likely to be the case – Google is just leveraging its online services to further promote Google+. But when you’re a company as big as Google, even minor changes like this can have large, even devastating, impacts in niche communities. In this case, a community that has very few social options left.

For what it’s worth, Brett Keller has created a petition to save the social features in Google Reader. It has over 4,000 signatures. That won’t be enough to get Google to reconsider Reader’s changes, we’d wager. After all, Google’s social future is wrapped up in Google+, for better or – in Iran’s case, at least – for worse.

With the tagline, “I am the Asshole %,” artist Ryan Casey puts some of entertainment’s biggest stars on blast.

Capitalizing on the ‘Occupy’ movement, Casey calls stars out for what he sees are their hypocrisy, materialism, and narcism. Featuring Beyonce, CNN’s Anderson Cooper, Chaz Bono, Kim Kardashian, Justin Bieber, Snooki, Lady Gaga, and others, the #OccupyHollywood comics takes some not-so-subtle shots at celebs.

But are they on point? You be the judge! 

 

What other stars should be on Ryan Casey’s #OccupyHollywood list?

The software patent system is totally askew.  We need to look no further to see this than the recent news that Apple was granted a patent on sliding to unlock a mobile device.  (Edit: It actually was granted back in February, but the case pinged again, and so we're all revisiting it.) It's bad enough that a governing body somewhere actually believes that you or I aren't smart enough to come to the natural conclusion on our own (that's basically what a patent means -- it's a unique idea or process), but the fact that it existed on an old Windows CE device in 2005 was totally overlooked.

I present the Neonode N1m, as reviewed by none other than Tnkgrl.

Youtube link for mobile viewing

Jump to 4:00 if you're impatient.  Now you might ask two things -- why was this patent granted, and does "prior art" really mean anything?  The first is an easy answer, the folks who granted the patent probably never heard of the Neonode N1m (but I'm sure Apple did).  You can't blame them if they didn't know, even if they should have.  The second question is a bit more tricky.  Usually, if prior art can be proven, a patent is invalidated.  Proven is a tough word that means more than one thing to different people.  It should be easy to interpret, but that's not how the legal system in the US works. 

It did work as expected in the Netherlands though, and Samsung has already brought the lowly Neonode N1m in front of the court there -- and had Apple's claims over slide to unlock determined to be "trivial and likely invalid", and the court refused to consider them.  I'm sure HTC and Motorola, who are being sued over multiple gesture patents in the US, already have this particular Youtube video bookmarked.  If this patent stands, we all should just go back to this.

More: 9to5Mac; Fosspatents

I’m going to use the longer posting capabilities (on here and Google+ which DOES NOT have RSS haha) to express my irritation about people removing RSS as a way to subscribe to websites.

Basically I’ve noticed a huge trend not only in websites moving away from RSS to Twitter and FB, but REMOVING IT COMPLETELY! Personally, I feel like this is NOT a good move for people who provide content to stay in touch with consumers. Why?

-Twitter has no good way to filter information sources and brands from friends. Lists are really not useable, I don’t think a lot of people use them, and the UI has them buried so it’s a pain in the ass to get to them. People are HIGHLY selective about who they follow, and they’re not gonna throw a dozen small blogs in with their best friends’ updates or the experience of using the service become greatly diminished.

-Facebook is a mess for trying to sort “likes” and pages, I don’t see how having a MASSIVE unwieldy feed of constant centralized updates helps ANYONE disseminate information. I would say you could check 5-6 Facebook pages a day MAX for blogs you like who have pages there. That’s a far cry from the 200+ blogs I check every day in about an hour.

RSS is a way to consume a LOT of information very quickly, and STORE it in nice categories if you miss it. So I can catch up with a small blog’s output at the end of the week and, if I so choose, read EVERY article easily in one sitting. You think on Friday I’m gonna go browse that same site’s Twitter feed on their page (digging through all the messy @ replies) and see what they did that week?! Or go to their Facebook page that is littered with contests? No way dude, I’m too busy for that!

I feel like small blogs cut their own throat by taking away the RSS capability. I give this analogy a lot, but social media outlets are INFO COLANDERS! 5% of your followers will see anything you post, and that’s probably only within 20 minutes of posting. That’s the way it is and it’s gonna only get worse. Apart from email lists, RSS is the best way you can collect stuff across the internet to read quickly, and I am so irritated when that choice is taken from me.
/rant

Washington is a city who appreciates a good sign. So when the 15 or so folks who got together to protest changes to Google Reader outside the Google's D.C. headquarters this afternoon, we figured that they might try and put their thoughts on a few pieces of posterboard. [ more › ]

The demise of Google Reader's share features is affecting everyone from RSS-junkies to Iranian freedom fighters, and many of them are very displeased. Google Reader itself is very much alive and well, and in fact, the RSS reader will soon sport a slick-looking redesign. However, at some point this week, the ability to follow, friend and share links within Reader will cease to exist, as Google pushes people to use Google+ for those kinds of things. Along with it, the myriad communities that depended on Reader for years for everything from meeting new people to organizing protests will just have to figure something else out. It turns out Google wasn't so bad at social networks, after all.

Displeased is actually not a strong enough word to describe these sentiment. These people are pissed, and they're fighting back. (There are already protests planned outside of Google offices and a petition--using Google Docs, of course--to save Reader's social features.) The dissenters divide into two camps: Iranians and the Sharebros.

Iran

You've undoubtedly heard a lot about how the Iranian protests of 2009 were driven by Twitter. They were, and the government responded immediately, imposing firewalls and other censorship measures in an effort to quash the uprising. Ultimately, pretty much every single social network was banned in Iran over the past couple of years, leaving Iranians without a free speech forum--except Google Reader. As an Iranian blogger who goes by Amir explains, "Google Reader is not in a separated domain (like any other Google product) and thanks to https protocol, it is hard to filter by government." Amir and other Iranians are worried that Google+, which lives in a subdomain that could be more easily blocked, will suffer the same fate as the other social networks:

Popularity map for Google Reader shows that Google Reader is the 1st popular website in Iran, despite the fact that many users which are using VPN or proxies and are not counted! Then it makes sense why Google Reader matters for Iranian and why integrating it with Google+ will makes it like any already available and banned website like facebook!

Alan Green, the Google programmer saddled with breaking the bad news on the company's official blog, has received nearly 500 comments on his Google+ post announcing the changes, many of them from angry Iranians who believe their freedom of speech is in jeopardy. He hasn't responded to a single one.

The Sharebros

Sharebros identify themselves simply as "person(s) whom one is following and followed by on Google Reader (as formally recognized by a Google Reader founder)," but their devotion to Reader is uncanny. Like more well known online communities like Reddit or 4chan, they've developed their own glossary of hashtags and lexicon of pictograms. ( -_-  denotes "dissatisfaction or defeat.") The community doesn't just exist online, either. They have #sharebro parties, and many of them say they've met some of their closest friends via Google Reader. At least one Sharebro met his wife.

We talked to a number of Sharebros about the changes, and their arguments against axing the social features were about half as compelling as their stories about how Google Reader changed their lives. Richard Berger told The Atlantic Wire about how he ditched email and now primarily communicates with his mother via Google Reader. "Reader is the best way to share and discover content on the internet, period," says Richard.

Another, Stu Watson formed a band, No Sky God, with his friend James who he met through Google Reader. They just put out a 12" record featuring Nat Baldwin, the bassist from Dirty Projectors. "James and I would likely have never met without Reader," Stu wrote in an email, "but more importantly the social aspects of Reader allowed us to get to know each other in a more intimate and ultimately meaningful way than if we had, say, just been Facebook friends or something." Stu tells us that people in the Sharebro sphere are already working on coding their own Reader replacement--they don't want to use Google+.

The most compelling Sharebro story comes from Ramey Moore. In a Reader forum post, Ramey tells the story about how he and his wife, who's expecting a child soon, "first met at a #sharebro event, scheduled and organized solely through Reader." The (long) post concludes: 

I am 100% certain that even if someone at Google cares or commiserates with me that nothing will change. … Basically, this is a panegyric. An elegy. A final funeral oration. A fuck you to those who want to kill something that works and try to build a Frankenstein's Monster that will probably choke to death on its own blood. Reader works as is, this move is just a sad attempt to jump-start an already failed Buzz 2.0.

That's it, I'm done. If anyone cares I'm going to be trying to imagine how to delete G+.

At least one protest organized by Sharebros is scheduled to gather tomorrow at Google's DC headquarters. Meanwhile, the petition to save Reader's social features has gathered thousands of signatures. And despite all the backlash from their most devoted users, the Google Reader teams has yet to respond. We reached out for a response and haven't heard back.

Hidden within Nokia's Future Lounge, this very flexible display offers up a glimpse of what sort of thing we could possibly be dealing with when we roll up to Nokia World in 2021. The prototype Nokia Kinetic Device, including its display, can be flexed across both the vertical and horizontal planes -- with bending and twisting motions controlling the interface. If you bend the screen towards yourself, it acts as a selection function, or zooms in on any pictures you're viewing. In music mode, you can navigate, play and pause with the tactile interface. It's still a way off from arriving on phones, though Nokia is aiming to whet developers' appetites with this prototype. We may have seen some twisty interfaces already, but nothing packing a four-inch screen and built-in functionality like this. Nokia couldn't confirm the screen technology being used. Could that be a flexible AMOLED display? See those impressive viewing angles and contortions after the break and judge for yourself.

Continue reading Nokia's kinetic future: flexible screens and a twisted interface (video)

Nokia's kinetic future: flexible screens and a twisted interface (video) originally appeared on Engadget on Wed, 26 Oct 2011 12:50:00 EDT. Please see our terms for use of feeds.

Permalink   |   | Email this | Comments

It happens all too often: Put a bunch of really smart people in a room, tell them to solve a problem, and watch as they dissolve into blathering idiocy.

Okay, maybe it’s not all that bad. But we’ve all seen groups of supposedly smart people who just can’t work well together. That’s because, according to recent research from Massachusetts Institute of Technology, Carnegie Mellon, and Union College, raw smarts doesn’t have much to do with team performance.

Remarkably apt.

Submitted by: probst
Posted at: 2011-10-10 06:13:40
See full post and comment: http://9gag.com/gag/341187

Earlier this week, we published a chart-essay that illustrates the extreme inequality that has developed in the US economy over the past 30 years.

The charts explain what the Wall Street protesters are angry about. They also explain why the protesters' message is resonating with the country at large.

Here are the four key points:

In this 3 parts post I want to show:

1. how standard, out-of-the-box, Scala helped me to code a small application
2. how Functional Reactive Programming brings a real improvement on how the GUI is built
3. how to replace Parser Combinators with the Parboiled library to enhance error reporting

You can access the application project via github.

I can do that in 2 hours!

That's more or less what I told my wife as she was explaining one small problem she had. My wife is studying psychology and she has lots of essays to write, week after week. One small burden she's facing is keeping track of the number of words she writes because each essay must fit in a specific number of words, say 4000 +/- 10%. The difficulty is that quotations and references must not be counted. So she cannot check the file properties in Word or Open Office and she has to keep track manually.

For example, she may write: "... as suggested by the Interpretation of Dreams (Freud, 1905, p.145) ...". The reference "(Freud, 1905, p.145)" must not be counted. Or, "Margaret Malher wrote: "if the infant has an optimal experience of the symbiotic union with the mother, then the infant can make a smooth psychological differentiation from the mother to a further psychological expansion beyond the symbiotic state." (Malher cited in St. Clair, 2004, p.92)" (good luck with that :-)). In that case the quotation is not counted either and we must only count 3 words.

Since this counting is a bit tedious and has to be adjusted each time she does a revision of her essay, I proposed to automate this check. I thought "a few lines of Parser Combinators should be able to do the trick, 2 hours max". It actually took me a bit more (tm) to:

• write a parser robust enough to accommodate for all sorts of variations and irregularities. For example, pages can be written as "p.154" or "pp.154-155", years can also be written "[1905] 1962" where 1905 is the first edition, and so on

• use scala-swing to display the results: number of words, references table, file selection

• write readers to extract the text from .docx or .odt files

Let's see now how Scala helped me with those 3 tasks.

Parsing the text

The idea behind parser combinators is very powerful. Instead of building a monolithic parser with lots of sub-routines and error-prone tracking of character indices, you describe the grammar of the text to parse by combining smaller parsers in many different ways.

In Scala, to do this, you need to extend one of the Parsers traits. The one I've choosen is RegexParsers. This is a parser which is well suited for unstructured text. If you were to parse something more akin to a programming language you might prefer StdTokenParsers which already define keywords, numeric/string literals, identifiers,...

I'm now just going to comment on a few points regarding the TextParsing trait which is parsing the essay text. If you want to understand how parser combinators work in detail, please read the excellent blog post by Daniel Spiewak: The Magic behind Parser Combinators.

The main definition for this parser is:

  def referencedText: Parser[Results] =
    rep((references | noRefParenthesised | quotation | words | space) <~ opt(punctuation)) ^^ reduceResults

This means that I expect the text to be:

• a repetition (the rep combinator) of a parser

• the repeated parser is an alternation (written |) of references, parenthetised text, quotations, words or spaces. For each of these "blocks" I'm able to count the number of words. For example, a reference will be 0 and parenthetised text will be the number of words between the parentheses

• there can be a following punctuation sign (optional, using the opt combinator), but we don't really care about it, so it can be discarded (hence the <~ combinator, instead of ~ which sequences 2 parsers)

Then I have a function called reduceResults taking the result of the parsing of each repeated parser, to create the final Result, which is a case class providing:

• the number of counted words
• the references in the text
• the quotations in the text

Using the RegexParser trait is very convenient. For example, if I want to specify how to parse "Pages" in a reference: (Freud, 1905, p.154), I can sequence 2 parsers built from regular expressions:

  val page = "p\\.*\\s*".r ~ "\\d+".r

• appending .r to a string returns a regular expression (of type Regex)
• there is an implicit conversion method in the RegexParsers trait, called regex from a Regex to a Parser[String]
• I can sequence 2 parsers using the ~ operator

The page parser above can recognize page numbers like p.134 or p.1 but it will also accept p134. You can argue that this is not very well formatted, and my wife will agree with you. However she certainly doesn't want to see the count of words being wrong or fail just because she forgot a dot! The plan here is to display what was parsed so that she can eventually fix some incorrect references, not written according to the academia standards. We'll see, in part 3 of this series how we can use another parsing library to manage those errors, without breaking the parsing.

One more important thing to mention about the use of the RegexParsers trait is the skipWhitespace method. If it returns true (the default), any regex parser will discard space before any string matching a regular expression. This is convenient most of the time but not here where I need to preserve spaces to be able to count words accurately.

To finish with the subject of Parsing you can have a look at the TextParsingSpec specification. This specification features a ParserMatchers trait to help with testing your parsers. It also uses the Auto-Examples feature of specs2 to use the text of the example directly as a description:

  "Pages"                                                                           ^
  { page must succeedOn("p.33") }                                                   ^
  { page must succeedOn("p33") }                                                    ^
  { pages must succeedOn("pp.215-324") }                                            ^
  { pages must succeedOn("pp.215/324") }                                            ^
  { pages("pp. 215/324") must haveSuccessResult(===(Pages("pp. 215/324"))) }        ^

Displaying the results

The next big chunk of this application is a Swing GUI. The Scala standard distribution provides a scala-swing library adding some syntactic sugar on top of regular Swing components. If you want to read more about Scala and Swing you can have a look at this presentation.

The main components of my application are:

• a menu bar with 2 buttons to: select a file, do the count
• a field to display the currently selected file
• a results panel showing: the number of counted words and the document references

count example, note that the parsing is not perfect since the word counts do not add up!

If you have a look at the code you will see that this translates to:

• an instance of SimpleSwingApplication defining a top method and including all the embedded components: a menu bar, a panel with the selected file and results
• the subcomponents themselves: the menu items, the count action, the results panel
• the "reactions" which is a PartialFunction listening to the events created by some components, SelectionChanged for example, and triggering the count or displaying the results

I was pretty happy to see that much of the verbosity of Swing programming is reduced with Scala:

• you don't need to create xxxListeners for everything
• there are components providing both a Panel and a LayoutManager with the appropriate syntax to display the components: FlowPanel, BoxPanel, BorderPanel
• thanks to scala syntax you can write action = new Action(...) instead of setAction(new Action(...))

This is nice but I think that there is a some potential for pushing this way further and create more reusable out-of-the-box components. For example, I've created an OpenFileMenuItem which is a MenuItem with an Action to open a FileChooser. Also, something like a pervasive LabeledField with just a label and some text would very useful to have in a standard library.

I also added a bit of syntactic sugar to have actions executed on a worker thread, instead of the event dispatch thread (to avoid grey screens), using the SwingUtilities.invokeLater method. For example: myAction.inBackground will be executed on a separate thread.

Eventually, I was able to code up the GUI of the application pretty fast. The only thing which I didn't really like was the Publish/React pattern. It felt a bit messy. The next part of this series will show how Functional Reactive Programming with the reactive library helped me write cleaner code.

Reading the file

I anticipated this part to be a tad difficult. My first experiments of text parsing were using a simple text file and I knew that having the user (my wife, remember,...) copy and paste her text to another file just for counting would be a deal-breaker. So I tried to read .odt and .docx files directly. This was actually much easier than anything I expected!

Both formats are zipped xml files. Getting the content of those files is just a matter of:

• reading the ZipFile entries and find the file containing the text

  val rootzip = new ZipFile(doc.path)
  rootzip.entries.find(_.getName.equals("word/document.xml"))
  

• loading the xml as a NodeSeq

  XML.load(rootzip.getInputStream(f)))
  

• find the nodes containing the actual text of the document and transform them to text

  // for a Word document text is under <p><t> tags
  (xml \\ "p") map (p => (p \\ "t").map(_.text) mkString "") mkString "\n"
  

For further details you can read the code here.

Recap

That's it, parser combinators + scala-swing + xml = a quick app solving a real-world problem. In the next posts we'll try to make this even better!

</status><status class="ok">

</status><status class="ok">