Daniel and I took a week off from Core Intuition for the holidays, but I was a guest on the iDeveloper Live podcast last week for an end of year show with Scotty, John Fox, Saul Mora, Brent Simmons, and Guy English. Topics included highlights from the year’s events, what projects we’re working on now, learning from Apple’s successes and failures, predictions for 2013, and some detours into Auto Layout, Core Data, iCloud, and other new APIs.

Here’s the episode on iDeveloper TV with the audio and links to all the guests.

Even though I’ve been podcasting for years, I think this was the first podcast I’ve been on that was actually broadcast live. It was a fun show. Check out the iDeveloper Live web site for previous episodes and to subscribe.

As 2012 was winding down, I was fascinated with LongPosts.com (built on top of App.net) and so used it to post some thoughts about Twitter archives. The site is gone now, so I’ve moved the text back to my weblog here, where it belongs anyway. The link to the ADN discussion is here. — future me, January 2016

One of the main goals of my web app Watermark is to archive and search tweets and ADN posts, so it was natural for me to implement support for Twitter’s new archive export format. I finished it last night and linked it from the Watermark account page this evening for all customers.

I had heard that Twitter’s export included a CSV version before I saw the actual files, so I started work coding an importer based on that, with the assumption that I could tweak it later. Once I saw a real tweets.zip, I had to throw out most of my initial work. The CSV files have two problems:

• They don't properly escape values using quotes, so a comma inside a tweet makes the files more difficult to parse.
• They don't include some essential Twitter metadata like the reply-to ID. I switched to using the JSON files and it's working well. They're JavaScript but not strictly JSON, so you just skip the first line.

Since the ZIP archive can be fairly big, instead of uploading in a web browser I let the user choose the file via Dropbox. This was a nice opportunity to try out the Dropbox Chooser. Then on the server I extract the files and load the data.

Dave Winer is doing something interesting with archives too. He’s started linking up other people’s archives on S3 — both the HTML view and the .zip file. I have a test Watermark account that I’ve loaded one of these into. It’s interesting to import multiple archives and have them all merged together and searchable.

For so long we’ve waited for access to our old tweets. In the meantime I’ve shipped two products around fixing this limitation, so it’s especially funny that Twitter finally rolls out archives after I’ve stopped posting there. (And of course I love that ADN has allowed access to your full post history from the very beginning.) Not entirely sure where all this is going to lead, but I agree with Dave Winer that new apps should be possible now.

Since introducing the Tweet Marker $1/month subscriber plan earlier this week, I’ve received a few questions about how the Safari extension works, and whether Watermark customers will also receive the new features. Yes, Watermark subscribers automatically have access to the Tweet Marker extension, which can be downloaded here.

I’ve prepared a screencast to show how the extension works. It’s about a minute long, and you can view it right here.

Thanks to everyone who has already subscribed to either Tweet Marker or Watermark.

The original goal for Tweet Marker Plus was to help cover the hosting costs for Tweet Marker. It succeeded for a little while, but it also ended up evolving into a larger independent service: Watermark, with much higher hosting costs for archiving and search, and a bunch of new features like App.net support, Dropbox sync, saved collections, and more. I’m really excited about the future of Watermark.

I also hear from Tweet Marker users who don’t need Watermark. They still want to support Tweet Marker, though, to make sure it continues running and that it’s as fast as possible.

So today I’m introducing a separate, inexpensive subscription option for Tweet Marker. Just $1/month! You can subscribe from the new Tweet Marker home page. And as a bonus you’ll get the first official Safari extension for Tweet Marker, shown in this screenshot:

Watermark now has App.net posting, replying, and an improved UI for tweeting. You could always tweet from Watermark, but it used Twitter’s “web intents” system, which opens a new browser window. The new interface is integrated directly into the Watermark sidebar, and it works with both App.net and Twitter.

Here’s what it looks like when it’s visible:

Watermark had some downtime early last week. While I was able to bring back the server faster and hopefully more robust than ever, I also wanted to quickly act to improve the service in visible ways. So I wrapped up a few features for Watermark over the Thanksgiving break, including two specific new features for Twitter and App.net.

For Twitter: Copying tweets to custom collections was cumbersome before, involving lots of clicks if you are copying multiple tweets in a row. Now there’s a faster way. After you copy a tweet to a collection, Watermark remembers that recent collection for a couple minutes and offers a “quick copy” link directly next to the tweet.

For App.net: You can now repost or star a post directly from the Watermark interface. I’ll continue to fill out Watermark with more features like this, whether you’re living in Watermark as your default client, or just searching your archive and want access to more functionality.

Search and performance should also be better across the board.

Last month I tried an experiment, lowering the price of Tweet Library for the first time in 2 years. It wasn’t selling well and I wanted to do something to determine if I was just stubbornly pricing it too high or if there was a deeper issue with the quality or marketing of the app. So let’s follow up on whether this was a success or not.

Here’s the graph of revenue for a 2-month period: one month before the price cut and one month after. There was also a new version released about a week after the price cut, but it didn’t appear to have a significant impact on sales.

Downloads were up 175%. Profit was up 40%. My gut feeling is that I should have dropped the price to $7.99 instead of $4.99, but I’m wary of changing anything again right now. We’ll see what next month looks like.

Watermark now supports App.net’s Stream Marker. You can click a post in Watermark to set its marker, the current marker will be retrieved when refreshing posts from the server, and the “Scroll to marker” link scrolls to the marker or loads more posts trying to find it.

Here’s what a marked post looks like:

I don’t know if any other clients support Stream Marker yet, but I expect many will in their next updates. I thought it was appropriate that I should be one of the first to support it.

I launched Tweet Marker in June of 2011. It wasn’t exactly a new idea — Echofon had a private sync before, and developers had long been asking Twitter for an official timeline position sync API — but Tweet Marker was the first to support sync between different apps. And it was the first to bring this idea to many more users, to make it a must-have feature that customers asked for.

In a way, I think of Stream Marker as a next-generation Tweet Marker, even though it’s not mine. The API and name are different, but it feels familiar, like an evolution from what we learned from Twitter apps. It’s a no-brainer to support it in Watermark.

Great essay from Rands on Scott Forstall. It’s one of the first I’ve seen to capture what made Forstall valuable to innovation at Apple:

“While I’d continued to hear about the disdain amongst the executive ranks about Forstall after I left Apple, I was still shocked about his departure, because while he was in no way Steve Jobs, he was the best approximation of Steve Jobs that Apple had left. You came to expect a certain amount of disruption around him because that’s how business was done at Apple — it was well-managed internal warfare. Innovation is not born out out of a committee; innovation is a fight.”

One part of this executive shakeup that had me puzzled was the rumor that Scott Forstall refused to sign an apology letter about iOS 6 Maps. We’ve had a few open letters from Steve Jobs, and now one from Tim Cook. It seemed out of character to have a VP do it, someone who’s lesser known to the general public.

But then I ran across this letter about EPEAT from Bob Mansfield again, posted just a few months ago. It is signed only by Mansfield. It starts:

“We’ve recently heard from many loyal Apple customers who were disappointed to learn that we had removed our products from the EPEAT rating system. I recognize that this was a mistake. Starting today, all eligible Apple products are back on EPEAT.”

In other words: we’re listening, we’re sorry, and here’s what we’re doing to set things right. And I think that’s Tim Cook’s Apple. Proud and passionate about the products they’ve built, definitely, but always sincere. Arrogance has no place.

Like many programmers, I’m often fooled into thinking that it’s enough to build a good product — that people will find it on their own, instantly recognize its value, and pay for it. It’s easy to forget that even great products need marketing to succeed. For a one-man shop it’s important to take a break from writing code and work on how the app is sold.

Building a business is hard. I started Riverfold Software 6 years ago and in many ways it has fallen short. And for some of the past year, I’ve squandered the success of Tweet Marker, failing to practice and experiment with how to make money from it.

Jason Fried of 37signals wrote for Inc Magazine last year about how making money takes practice:

“One thing I do know is that making money is not the same as starting a business. For entrepreneurs, this is an important thing to understand. Most of us identify with the products we create or services we provide. I make software. He is a headhunter. She builds computer networks. But the fact is, all of us must master one skill that supersedes the others: making money. You can be the most creative software designer in the world. But if you don’t know how to make money, you’re never going to have much of a business or a whole lot of autonomy.”

In the last week I’ve taken a couple steps in the right direction. I’ve finally redesigned the Watermark home page around a simpler marketing statement of what the app is about. And as discussed on the recent Core Intuition, I switched from PayPal to Stripe in an effort to make payment smoother and subscriptions easier to track. There’s still a lot to do, but I hope to make even more time for marketing before the year is up.

From John Gruber’s iPad Mini review:

“If the Mini had a retina display, I’d switch from the iPad 3 in a heartbeat. As it stands, I’m going to switch anyway. Going non-retina is a particularly bitter pill for me, but I like the iPad Mini’s size and weight so much that I’m going to swallow it.”

As I said on the last Core Intuition, it’s even more of an easy switch for me since I never upgraded to the iPad 3. The iPad Mini has essentially been my only iPad for the last week. I’m using it more, and taking it places that I would’ve have bothered with before. My new favorite Apple device.

The second issue of Marco Arment’s The Magazine is now available, featuring essays by John Siracusa, Gina Trapani, Lex Friedman, Daniel Rutter, and Alex Knight. In the new issue, Marco writes about the risk and success of the app:

“It was a risk to release an app that can only exist in Newsstand, full of paid writing, to an audience that’s predominantly people who have a surplus of free reading material and bury their empty Newsstand folders on their last screen of apps.”

I’ve been happy with how well The Magazine fits into my mobile reading workflow. I read a lot in Instapaper and Reeder, most of it technology-related articles. The Magazine occupies a more leisurely, thoughtful space for reading, away from the frenzied pace of everyday tech news. And because it’s a handful of essays just every other week, I don’t expect to ever be overwhelmed with it in the same way that it’s easy to fall behind in never-ending RSS and Twitter feeds.

High quality, highly recommended. Congrats to Marco on the successful launch.

I rolled out a small but powerful feature last night for Watermark. For a while you’ve been able to create saved filters, which are just shortcuts to quickly run a search across the Watermark database for your account. Saved filters are also cool because they automatically sync as CSV files to Dropbox. Now you can allow any of these saved filters to be shared with others.

Click “Allow saved filters” and you’ll get a link option next to each filter. That will produce URLs that can be posted to Twitter or App.net or wherever, and anyone can see the results of the search even if they don’t have a Watermark account.

It’s a way to expose a slice of your timeline and archive to other people. Here are a few that I’ve set up:

• cingleton — Every tweet or post mentioning the Çingleton conference.

• pdx food — Neven Mrgan’s short reviews of Portland restaurants.

• ipad mini — Obvious query given today’s Apple event.

All of these search live across my 6-month archive of about 275,000 tweets. Remember that Twitter’s own search only goes back about a week. There’s really no other way to get this kind of data.

It’s been over a year since I launched Tweet Marker, and I’ve been thinking a lot lately about what its second year will look like. Hosting costs have gone up significantly over the last year, though I’ve offset that by combining the hosting for Watermark and Tweet Marker together so that they share the same core servers. I’ve considered other options, too: run a Kickstarter-like fundraiser, charge developers, or ask for donations again.

Now that I’ve stopped posting to Twitter from my personal account, people also ask whether I’ll just shut Tweet Marker down. The answer is no. I’ll keep it running, even if it means funneling some revenue from Tweet Library and Watermark to pay for it. Even if it means having to put out fires and deal with other web hosting distractions.

Hosted web services have a different level of commitment than traditional apps. When I stopped selling Wii Transfer, existing customers could continue to use it for as long as they wanted. Not so with something like Tweet Marker, which becomes useless the minute I shutter the web server.

The Macworld Eddy statue sits on my desk as a constant reminder that real people like this thing and find it useful. There is a lot of uncertainty with Twitter, App.net, Tent.io, and the future of microblogging, but no matter what takes off and what sync looks like, I think Tweet Marker played an important role in the evolution of Twitter clients. I’ll always be proud of that. It would be a disservice to my customers and the Macworld award to ever consider turning off the API while people value it.

I have a new iPhone app in the store: Watermark Mobile, a lightweight companion app to Watermark, my search and archiving tool for Twitter and ADN. It’s free for existing customers, or $4.99 using in-app purchase to subscribe as a new Watermark customer.

With this app I wanted to solve two problems:

• Clean, simple search interface on the iPhone.

• Allow paying for Watermark inside the app with your iTunes account.

While I’d eventually love to have a more full-featured client like Tweet Library available for Watermark, after a quick weekend of hacking I decided that Watermark Mobile was already useful enough that I should release it. So I did.

At Çingleton last week, Michael Jurewitz talked about app pricing and the arguments for raising your price. He made a convincing case, and it echoed some of the themes that I wrote about before I released Tweet Library 1.0 back in 2010.

In the two years since, I never once changed the price. No intro discount, no gimmicks, never on sale; $10 was essentially set stone. Even as it moved to the iPhone as a universal app, I stuck to my original philosophy about pricing, perhaps stubbornly. There’s value in consistent pricing, so that the user knows what to expect from one month to the next, and to indicate that the developer attaches a specific value to the app.

Last week, before Çingleton and right as version 2.1 of Tweet Library was about to be released, I decided to try an experiment: I cut the price in half to $4.99. Even though it’s a niche app that only doubles as a full Twitter client, this puts it more in line with other Twitter apps on iOS. (And even cheaper than buying both the iPhone and iPad versions of some apps, like Netbot for ADN.)

Meanwhile, Tweetbot for Mac is now out at $20. Daniel Jalkut covers this on his new blog, Bitsplitting:

“Is $20 a reasonable amount to pay for Tweetbot? I think so. But if Tapbots would have preferred to charge even less, has it been fairly priced? Many folks are seizing on the coincidence of Tapbots needing to charge more as an opportunity to exalt ‘fair pricing,’ when this was a result of coercion in two directions.”

Pricing is something I am still very fascinated by, especially this constant pull between how we value our own software and how pragmatic we want to be as a business. I’m going to let Tweet Library sit at $4.99 for a month, and if revenue is not obviously greater than what it would have been, I’ll bump it back to $10 or a middle-ground $7.99.

Following just a week after the Dropbox support in Watermark, I’ve added two new smaller features that improve the user experience. The first is that photos hosted by Twitter are now included as inline thumbnails next to a tweet, as shown here:

The second improvement is a much faster Favorites view, which also now includes the new “stars” feature from App.net. Watermark doesn’t yet download all your existing stars (that will be rolled out to more users soon), but when it does encounter a star in your App.net timeline, that starred post will be included it this view right alongside Twitter favorites.

I’ve been making Watermark better. Sometimes it’s small tweaks or bug fixes; other times, more noticeable new features. Because it’s a subscription, I’m determined to improve it quickly and often. I don’t write about most of these changes, but the new Dropbox sync in Watermark deserves special attention.

Watermark originally shipped without any kind of export feature. This was a glaring omission for an archiving tool. But because of the large number of tweets stored by Watermark — some users have hundreds of thousands of tweets from their friends in the app — a simple export wasn’t feasible. I could have offered an export of just your own tweets, but then you also have the fairly clunky step of waiting for the server to gather tweets together, then downloading a file from your web browser, finding where to store it or the previous downloaded copy to replace it with.

Dropbox sync fixes that. Watermark can now automatically copy tweets (and App.net posts) from your saved filters and custom collections to CSV files on Dropbox. For example, search Watermark for “iPhone 5”, click “Save as filter”, and the most recent 1000 tweets matching that query will appear in a file called “iPhone_5.csv” on Dropbox. It keeps running in the background, so the files are updated every hour as new tweets matching the search are downloaded by Watermark, even if you aren’t signed in.

See the account page and FAQ for details and a sign-in link to authorize Watermark with Dropbox.

I’m renaming Tweet Marker Plus. Its new name — to better reflect its gradual move away from Twitter and syncing — is Watermark.

As part of the relaunch it immediately gains a new feature: App.net posts. You can now add an App.net account and it will download any posts from your friends, making them available for search. Watermark is already storing tens of millions of tweets, and I’m excited to start adding App.net posts to that archive as well.

So what happens to the basic Tweet Marker API? For now, nothing. The sync API that over 22 Twitter apps support will still be called Tweet Marker and remain Twitter-focused. Think of Watermark as a separate app: a new kind of client and archive tool, independent of Tweet Marker.

Justin Williams on his decision to stop selling his apps MarkdownMail and Today:

“Financially, it may not have made much sense to cut off the revenue streams, but therapeutically I’m freeing up that portion of my brain to focus my full attention on the next version of Elements and the dozens of other ideas that that are circling in my head.”

I felt exactly the same way when I stopped selling Wii Transfer earlier this year. It wasn’t until a month later that I realized how much I had been enjoying that revenue, limited as it had become. I don’t regret it, though. It was the right thing for my potential customers, not to be misled into thinking there would be new versions. And it was the right thing for my focus, working on other projects.

Recently I found myself in rare disagreement with Matt Gemmell:

“Indie devs are an endlessly replenishable resource. Good indie devs are similarly replenishable. This acquisition has no effect whatsoever on the rest of us, except for further legitimising the practice of big companies buying us up. That cannot possibly be a bad thing.”

While the general argument that Matt makes is solid — that Sparrow customers should be happy for the developers being acquired by Google, and that paying $3 for an app doesn’t give anyone a right to complain or feel betrayed — there are a couple ways that this acquisition could be a bad thing for everyone else.

Good indie devs, especially successful ones, are a limited resource. There are very few indie companies able to make a client as polished as Sparrow was, and even fewer with commercial success.

And with Twitter’s latest anti-competitive moves, we may end up losing another market that was friendly to indie developers and rich with UI innovation.

My favorite take on the Sparrow acquisition and what it means for sustainable indie software came from Rian van der Merwe:

“We need to reframe this argument. The real issue is much deeper than this specific acquisition. The real issue is the sudden vulnerability we feel now that one of our theories about independent app development has failed.”

The Sparrow acquisition came as a surprise to most of us. One day, they look like a successful company, taking on a difficult market and winning against free competition. The next day, they’re gone. I wish them luck at Google, but it is a loss for the community of small Mac and iOS companies.

(Speaking of Matt Gemmell, he’s just released a new Mac app called Sticky Notifications for sticking reminders in Mountain Lion’s notification center.)

With every day since Twitter’s new rules were announced, my opinion grows stronger that this is the end of the platform as we’ve known it for the last few years. It hurts that so much of what was pioneered by the developer community has been co-opted and trademarked and turned against developers. Nothing will change tomorrow; we can expect new versions of Tweetbot, Twitterrific, and my own app Tweet Library. But in the long run it’s a dead-end.

Lex Friedman, writing for Macworld:

“Some developers are notably hesitant to speak on the record, lest they incur Twitter’s wrath; the fear seems to be that since Twitter is now exerting more control than ever over access to its API—which developers leverage to make their Twitter apps work—that irking Twitter too much might result in a developer’s API access getting revoked.”

I have less at stake than developers with popular mainstream apps, so in a way I feel it’s my duty to speak out when it’s warranted, where others can’t. I’m quoted in Matthew Panzarino’s article on the Twitter API change, and I was interviewed for a radio spot on San Francisco’s KQED.

I could live with some of the API changes. I could live with the display requirements, even the user caps. Maybe I could live in a developer-hostile environment, finding a market niche far below Twitter’s radar. Except for two things:

• “Timeline integrity”: The display guidelines say you can’t mix other content in a timeline of tweets. You can’t easily have a multi-platform app that shows both App.net posts and Twitter tweets in the same view.

• Trademarks: You are allowed to use “Tweet” in the name of your app, but only if Twitter is the only service you support. You can’t add Facebook or App.net or Heello support alongside Twitter.

These are anti-competitive and unworkable. So I’ve decided to spin off Tweet Marker Plus, my paid web-based app that builds on top of the Tweet Marker sync engine. I’ll relaunch it with a new name, slowly introducing new features that aren’t tied exclusively to Twitter. For a hint of where it’s going, you can follow me on App.net.

Dave Winer reiterates that we must plan now to preserve our online writing:

“There were far fewer bloggers. Maybe thousands. Today there are millions. None of them are thinking about what happens when Tumblr or Blogger or WordPress or Facebook disappear. But come on — we almost know for certain that one of them will. Given enough time they will all disappear.”

These companies are only as strong and permanent as their leaders, and leadership doesn’t last. If you think it can’t happen soon, look at the new Digg. Although they want to export the previous content, currently nothing from Digg’s 7-year history is accessible. Not by accident, not by catastrophic failure, but because no one at the new company cared to keep it around.

Amy Hoy writes about master craftsmen and being accountable to your real customers:

“When you live and work in an insulated life — divorced from the end result of your work — you are spoiled. You’re graded more on your ability to please and manage gatekeepers than your work product. Gatekeepers are human; humans can be persuaded to accept excuses.”

Just enough separation from customers is healthy — email and tweets, instead of phone calls. But put up too many walls, too much bureaucracy, and you might no longer care who you’re building the software for. You might forget why it has to be great.

Daring Fireball turns 10 years old today. I love this visualization of the posts from those years. You can view by article length and highlight posts for certain topics.

There’s a rich history of posts in the archive. Like the best blogs, there’s consistency in design, tone, and format. None of the URLs have ever changed.

Here are some of my favorite essays.

June 4, 2004, Broken Windows:

“Arguing that it’s technically possible that the Mac could suffer just as many security exploits as Windows is like arguing that a good neighborhood could suddenly find itself strewn with garbage and plagued by vandalism and serious crime. Possible, yes, but not likely.”

April 20, 2006, Initiative:

“What I’ve concluded, though, is that if I want to make a full-time income from Daring Fireball, I need to just do it full-time. I.e. that it’s not going to work the other way around — to wait for the revenue to burgeon and then start putting full-time effort into it.”

August 4, 2006, Highly Selective:

“I’m sure there are other examples of Mac apps that offer anchored list selection, but the point remains that the vast majority of software now follows Apple’s lead and uses the unanchored model for list item selection. If ‘Mac-like’ means ‘what most other Mac software does’, then in this case the Mac-like behavior is wrong, or at the very least, worse.”

October 2, 2008, The Fear:

“But this pitch also worked because it was true. All three of those products sound good on their own. All three in one device sounds insanely great. Jobs was introducing the iPhone simply by describing precisely what it was.”

April 24, 2009, Twitter Clients Are a UI Design Playground:

“I read web sites and email and RSS feeds on my iPhone, but Twitter is the one service where reading on my iPhone doesn’t feel constrained compared to reading on my Mac.”

June 26, 2009, Copy and Paste:

“That we had to wait two years for the iPhone’s text selection and pasteboard is a good example of one aspect of the Apple way: better nothing at all than something less than great.”

January 27, 2010, The iPad Big Picture:

“Software aside (which is a huge thing to put aside), it may well be that no other company could make a device today matching the price, size, and performance of the iPad. They’re not getting into the CPU business for kicks, they’re getting into it to kick ass.”

August 24, 2011, Resigned:

“The same thought, care, and painstaking attention to detail that Steve Jobs brought to questions like ‘How should a computer work?’, ‘How should a phone work?’, ‘How should we buy music and apps in the digital age?’ he also brought to the most important question: ‘How should a company that creates such things function?’”

December 25, 2011, Merry:

“— how much will I be willing to pay then to be able to go back in time, for one day, to now, when he’s eight years old, he wants to go to movies and play games and build Lego kits with me, and he believes in magic?”

February 16, 2012, Mountain Lion:

“This is an awful lot of effort and attention in order to brief what I’m guessing is a list of a dozen or two writers and journalists. It’s Phil Schiller, spending an entire week on the East Coast, repeating this presentation over and over to a series of audiences of one.”

Congrats John. Here’s to the next 10 years.

Today, App.net passed its $500,000 funding goal. A few weeks ago when I signed up with my $50, I didn’t think they could do it. And Daniel and I were both pessimistic about their chances when we talked about it on Core Intuition 50.

In less than a month, they went from a mission statement video that seemed just a step away from vaporware, to following through on an API spec and then alpha version web site. They delivered. The momentum of shipping something real brought in new users and drove them to the finish.

What I love most about App.net is the transparency. Founder Dalton Caldwell is a blogger, like one of us. Where we only hear from Twitter’s CEO, Dick Costolo, through big news publications or at conference keynotes, for Dalton we hear it directly from his own blog posts, the way a small company should communicate. Being on the ground in posts and tweets is a perfect complement to his goal of treating users and developers as real customers.

App.net will never overtake Twitter. Look no further than hashtags all over the Olympics as proof of that. But App.net can put pressure on Twitter to respect third-party developers, and with thousands of paying customers, all with a vested interest in making App.net something worthwhile, App.net has already surpassed every other Twitter clone that has tried and failed to build a community.

From Paul Graham’s essay on ambitious startup ideas:

"The way to win here is to build the search engine all the hackers use. A search engine whose users consisted of the top 10,000 hackers and no one else would be in a very powerful position despite its small size, just as Google was when it was that search engine. […]

“Don’t worry if something you want to do will constrain you in the long term, because if you don’t get that initial core of users, there won’t be a long term. If you can just build something that you and your friends genuinely prefer to Google, you’re already about 10% of the way to an IPO, just as Facebook was (though they probably didn’t realize it) when they got all the Harvard undergrads.”

He’s talking about search engines, but it could be anything. Get those 10,000 passionate users and you have a chance to take on the giants in the industry. As of this writing, App.net has 8000 paying customers. And 25% of them signed up at the developer tier. I’m sure every developer with a popular Twitter app has already looked at the App.net API documentation.

As John Siracusa tweeted after App.net successfully funded: “Now comes the hard part.” Totally true, but just reaching this point was difficult — a perfect mix of great timing and even better execution. In the first 30 days, we saw a team that knows how to win. Let’s see what they can do next.

Shawn Blanc reviews the latest version of Day One, which now supports photos:

“Over the years, most of the major, monumental milestones of life were documented in my Moleskine. But not all. And that’s why I’m glad to have an app that let’s me easily and joyfully add a snapshot or a quick note about an important or memorable event. These are the things my family and I will look back on 20 and 30 years from now with great fondness.”

While I keep the important stuff in my journals, I also use a protected Twitter account for the everyday notes and photos while away from the house. It has no followers; it’s just to have a date-stamped entry with a photo that’s easy to sync. Now that I’ve read how people are using Day One for this, I’m going to switch away from my private Twitter account to use Day One on the iPhone instead.

I like having one place for this kind of stuff. If the same type of content is scattered across multiple services, it makes it less likely that everything will be together in the future when I finally want it.

Especially interesting to me from Shawn’s review is that he also keeps a hand-written journal, even after using Day One for a similar purpose. I’ll keep using real-world pen and paper too, and everything I write there I will also transcribe into Day One. But I’ll write new things in Day One that will stay exclusively digital.

Federico Viticci also has a great review. He starts with the big picture, the why of writing it all down:

“I don’t even know if I’ll be around in twenty years. But I do know that I want to do everything I can to make sure I can get there with my own memories. We are what we know. And I want to remember.”

I think the best writers know that it matters what their work looks like in a decade, or two decades, whether the writing is private or public. You can see it in everything from permanent URLs to blog topics to what software they use — a conscious effort to create content that lasts.

Steve Corona on keeping a journal:

“And for the past 1091 days, I’ve been journaling every single day — that’s about 3 years or 12% of my entire life. My only regret? I wish I started sooner.”

When I was younger, I had tried off and on to keep a sketchbook or journal, but it never quite stuck. Like blogging, or writing, or drawing, or anything you aren’t paid to do, it takes setting a routine. There’s always something more important to do.

Then in 1998, I started a journal again with a renewed commitment. I filled a book from that day up until I got married. Then another book through when my daughters were born. Another for the first 10 years of their life, and my son’s. The travel, the big life moments, the election, the work. It’s not everything — sometimes the entries are every day and sometimes months go by with nothing — but it’s the stuff that matters, and the snapshots in time of little everyday things too.

I would be devastated to lose these books. Open the pages and it rolls back the years like a time machine, to a previous life full of small details that are priceless today. I’m writing the books half for my terrible fading memory, and half for my children, who will only care what these years have been like when it’s too late to ask me.

So I’ve recently started transcribing the handwritten entries into digital form. One page at a time, into Day One, then exported as plain text. It’s a long and tedious process, but multiple copies are the only sure way to make something last.

Last year I had to migrate the news blog section of the Staple! site from an ancient version of Movable Type (version 2.x) to Blogger. Even though Blogger has recently dropped features and seems mostly deprecated in favor of Google+, for this site there were a ton of existing users on Blogger. Upgrading just made sense.

However, what a file format mess. Export in Movable Type’s custom text format; import in Blogger’s Atom format. So first step is to find a service that’ll convert between the two, then manually fix up usernames so it imports properly. I exported, tweaked, and imported this file at least a dozen times before getting it right.

I was so frustrated because this wasn’t just accidental bugs. Developers made conscious choices that led us to this compatibility dead-end. They bet against Dave Winer and lost at a pivotal time in the development of blogging.

We had a format that was perfect for both blog syndication and as an interchange format between systems: RSS. Instead, some developers criticized RSS, then proceeded to create new products that have not been well cared for.

That is now part of their legacy. 10 years after blogging went mainstream, the end result of reinventing the wheel isn’t better software, it’s user frustration trying to get anything to work together.

This lesson keeps playing out, as if we’re doomed to repeat it with each new generation of file formats. Here’s this week’s post from Eran Hammer declaring OAuth 2.0 a failed format:

"The web does not need yet another security framework. It needs simple, well-defined, and narrowly suited protocols that will lead to improved security and increased interoperability. OAuth 2.0 fails to accomplish anything meaningful over the protocol it seeks to replace."

If you have a choice, always pick the old boring format that works above the new hotness that is only theoretically better.

This is kind of a short, technical footnote to my last essay. There I linked to an older blog post from Dave Winer, just one of many of his on this subject. Today he writes how we should archive blogs before we worry about Twitter:

“With Twitter there’s a rich corporation minding it. They can and imho should be funding their own archive. But with the historic blogosphere, dating back to the early-mid 90s, a lot of it is already gone. The need to preserve it, by independent historians and librarians, is greater than the need for Twitter to be publicly archived.”

I have a lot to say on this, and I can’t wait to share a new web project that I started recently which could play a small roll in blog backups. When I killed off my little app Wii Transfer, I did so to refocus Riverfold around preservation. I wrote:

“It also doesn’t fit into a new theme I have for Riverfold: apps that are all about keeping and remembering what matters. For Clipstart, that’s family videos. For Tweet Library and Tweet Marker Plus, that’s old tweets.”

Dave mentions libraries several times in his blog post. It’s no accident that the word “library” is in Tweet Library’s name; my ambition for this app far outpaces my coding speed. But blogs are a different problem, and they need something special — perhaps multiple solutions.

Nothing lasts on the internet. I could write on my weblog for years and the next day get hit by a bus. The domain expires, the posts are lost, and it doesn’t matter if I had 10 readers or 10,000; it’s as if it never happened.

I love real books. I keep flirting with attempts to catalog our bookshelves over the years. My daughter offered to help once, excited through the first hundred books before she realized the rest would take all day and lost interest.

Some people say “good riddance” to the cheap printed book, but I don’t agree. Recently in our house I found a paperback of an old favorite, Tigana, which I had bought while traveling in Europe. Inside the cover I had written “Oxford, 1999”. I flipped through the pages and out fell a wine label that I hadn’t seen in 13 years. It was from a bottle of wine my wife and I had in Greece, sitting on the sand of an island beach the night I proposed.

I had kept it back then because I knew years later it would matter — a memory fused into a piece of paper, waiting. That trip was a story told in events like that one, in personal journals, and through email to family. The digital parts of the story didn’t last; the email is gone.

Write on Twitter and it vanishes from the internet after 3200 more such posts, unlinked and unfindable. But write the same on a scrap of paper tucked into a book and it will be rediscovered again years later.

A self-published novel in PDF on your web site is a ticking time bomb, waiting for your hosting bill to go unpaid. But print 10 copies and give it to 10 friends and it lasts forever.

The only way to preserve something is to make multiple copies and distribute them. The problem with digital is that it makes it just as easy to accidentally delete or lose copies as it is to create them. Evolving file formats and storage devices require constant supervision and maintenance, pushing files up each technology bump from floppies to CDs to Zip disks to DVDs to hard drives. It never ends.

We need to solve this. It’s something Dave Winer has written about. It’s something anyone with a large collection of writing online probably thinks about. How do we preserve the culture and art and stories of our time when the preferred media is so fragile?

Despite everything I said about how easy it would be for customers to upgrade from the Mac App Store version of Clipstart to the direct download version, in the real world this doesn’t appear to be working well for some customers. Might be a little buggy, and it’s too late to fix anything in the Mac App Store. So I’m doing upgrades the old-fashioned way, giving everyone new serial numbers.

After the first few emails came in, I automated this with a simple form that customers can fill out. It’ll give them a new serial number right away and email the registration information. I’ve added a link in the FAQ on the Clipstart home page too.

Marco Arment has a strong post on how sponsorships won’t change his writing:

“If a sponsor ever has a problem with something I write, and that affects pending or future sponsorship buys, that’s fine with me. I can find other sponsors. And if I can’t, I’ll write for free, like I did for years.”

This was part of a theme on today’s Core Intuition episode as well, not just sponsorships but also whether having an audience changes how we write tweets and blog posts. Daniel Jalkut and I have been very lucky with Core Intuition so far to have great sponsors: apps and events we already love like Marco’s own Instapaper, Smile’s TextExpander, CocoaConf, iDeveloper TV, NSConference, and Glassboard. Eventually we’ll be approached by more apps and services that we don’t have as much experience with, but as long as we keep our voice and honesty, I’m not too worried.

Gabe at Macdrifter comments on the shift back to direct download for 1Password:

"I recall the massive forum discussion about the decision to take 1Password MAS only. I converted to the MAS version in March to get on-board with their product roadmap. Now I see that it is available again as a direct download purchase and @roustem confirms it will receive the next update soon."

I’ve never liked the idea of being exclusive to the Mac App Store. I don’t think any of those transitions — from 1Password to Pixelmator — were good for users, especially when customers had to re-purchase a product they already owned. I hope sandboxing will at least make more developers think twice about pinning their business to the Mac App Store.

The blog post continues with this point, and all the good and bad that comes with it:

"This is the MAS as I see it: It is not for utility apps or power users. Apps like 1Password, TextExpander and Keyboard Maestro should all be purchased directly from the developers site. The MAS is for people afraid of using a computer."

In other words: it’s good for some users and some apps, and not others. And that’s okay. I have no regrets about pulling Clipstart from the Mac App Store. I think of my app as pretty easy to use, but it’s for people who get file systems and tagging and uploads. Most definitely not for people afraid of using a computer.

I’ve written about saved filters in Tweet Marker Plus, and now I’m happy to announce the latest new feature: favorites. Tweet Marker Plus now grabs your favorites from Twitter so that they’re included in the searchable archive. The UI is better too, so you can tell at a glance what tweets have been favorited, and a new sidebar link can show just your recent favorites.

A few days ago was the 3-month anniversary of Tweet Marker Plus’s launch. This is a significant milestone for me because all the early subscriptions were only billed once every 3 months. For most subscribers, this week is the first time recurring billing will have kicked in. (New subscribers are on monthly billing, which is a lot simpler for customers to understand and for me to predict.)

I also rolled out some other fixes tonight, and improved performance for how background tasks run. Enjoy.

It usually takes a couple problems hitting at once to cause a major server outage. This happened last week when Tweet Marker’s SSL certificate expired. I have the SSL set to auto-renew, but it still requires manually installing the new certificate, and other problems happened along the way.

First mistake: I didn’t realize it was expiring. Those emails go to an account I don’t check very often, littered with spam. And the email to confirm the renewal went to yet another email address that no longer worked. When I had moved the DNS hosting to Amazon’s Route 53, I had neglected to move over the MX records.

After fixing all of that, I tried updating the app on Heroku to use the new cert, only to get stalled as Heroku’s new SSL add-on rejected it. Certain I had done something wrong, I fumbled through a dozen Heroku SSL how-to posts before finally reverting to their old SSL add-on. It’s no longer documented and is in fact actively discouraged by Heroku, but it also has the lucky trait of actually working with my certificate. Updating DNS caused another hour-long delay because of the high TTL.

I sent two support requests during this process, so I thought I’d rate how each company did:

• DreamHost: Before I figured out the bad email address, I sent DreamHost a question about why the SSL certificate hadn't showed up yet. They responded very quickly, and even included a "P.S." that they were fans of Tweet Marker. Basically they provided excellent support, the best you could ask for.

• Heroku: When the new SSL add-on wasn't accepting my certificate, I filed a support request with Heroku as well. The response was an automated reply that they don't do support past 6pm. For a hosting company that charges a premium, this was a disappointing response. (They responded first thing the next morning, though.)

This SSL glitch was the only significant outage Tweet Marker has had in its first year. I learned a few lessons, took the opportunity to check backups and EC2 servers, and now I'm ready to move on. Hoping for an even better year 2.

Excellent piece from Joel Spolsky yesterday on software inventory and bug databases:

“The trouble is that 90% of the things in the feature backlog will never get implemented, ever. So every minute you spent writing down, designing, thinking about, or discussing features that are never going to get implemented is just time wasted. When I hear about product teams that regularly have ‘backlog grooming’ sessions, in which they carefully waste a tiny amount of time and mental energy every day or every week thinking about every single feature which will never be implemented, I want to poke my eyes out.”

Reminds me of the original Getting Real from 37signals. Back in 2006:

“So what do you do with all these requests that pour in? Where do you store them? How do you manage them? You don’t. Just read them and then throw them away.”

And:

“It sounds blasphemous but the ones that are important will keep bubbling up anyway. Those are the only ones you need to remember. Those are the truly essential ones. Don’t worry about tracking and saving each request that comes in. Let your customers be your memory. If it’s really worth remembering, they’ll remind you until you can’t forget.”

When I launched Tweet Marker Plus, I documented that it would store “about a month” of tweets. I didn’t want to promise too much before I fully understood the storage requirements. After a few weeks, I officially bumped it up to “at least 2 months”. I also added a full archive of your own tweets, which are never deleted.

The truth is I’ve yet to write the code that actually deletes any tweets from the database and search index. Eventually I’ll have to, but not yet. So I’ll continue to evolve the service in a way that makes it more useful and sustainable.

Recently I increased the $2/month price to $5/month, with the search index expanded to 3 months of tweets. Today I’m officially bumping the storage again, to 6 months of tweets. I’ve also changed to monthly billing instead of once every 3 months. Everyone who already has the $2/month plan will get to keep it. No price increase for you, and you still get the new 6-month storage and new features, as a thank-you for being an early subscriber.

And I’ve added a major new feature. You can now create custom collections of tweets and publish them to share with others. This is a feature from my iOS app Tweet Library, and in fact any published collection from Tweet Library will also show up in Tweet Marker Plus. New in Tweet Marker, you have the option of keeping a collection private or making it linkable, without it showing up when someone browses the list of collections.

The screenshots linked on the account page include an example of how collections work.

As a bonus for Mac users, there’s also a new menubar search app. This little Mac app hides in the menubar and gives quick access to searching your Tweet Marker Plus timeline and archive. Here’s a screenshot of what version 1.0 looks like.

Today I released Clipstart 1.5 for direct download customers and removed the previous version of Clipstart from the Mac App Store. Even though I’ve written about leaving the Mac App Store several times, actually pulling the trigger was difficult. But I believe it’s the right thing for my app, right now.

Clipstart 1.5 is still $29. Changes for this release:

• Now requires Mac OS X 10.7 Lion. Please do not upgrade to this version if you are running 10.5 or 10.6.

• Added "Consolidate" command for finding videos that have not been imported or duplicates outside the library.

• Added AppleScript support for uploading videos. See the help menu for scripting examples.

• Improved performance throughout the app, especially with importing and refreshing video thumbnails.

• Improved error message when failing to import all videos from a camera.

• Fixed tags field on Add Folder dialog not resetting tags value after an import.

• Support for Mac OS X 10.8 Mountain Lion's Gatekeeper security.

If you've been thinking about buying Clipstart but were on the fence before, I'll throw in a discount for you. Use coupon code MANTONBLOG to save 20%.

Since I follow Enrico Casarosa on Twitter, I’ve been hearing about La Luna for what seems like a year. I was so happy to finally see the film run in front of Brave last weekend. Beautifully done and possibly my favorite Pixar short, perfect for a short film. (As an added bonus for seeing it at Alamo Drafthouse, they ran 7 other Pixar shorts before the feature started.)

There’s so much interesting work possible with short films. Disney apparently has another great one up their sleeves in Paperman. Some of the early hype:

"From what they told me and from what I've glimpsed in these concept drawings and videos below, it could be a game-changer for hand-drawn animation. In fact, it's a new holistic hybrid of 2D and 3D, thanks to an interface called Meander that they 'hijacked' for the project from a young Disney programmer named Brian Whited, which was then refined by animator Eric Daniels and others. It's a vector-based tool that empowers the animator with digital in-betweening and puts the drawings back on the screen."

Also check out the series of YouTube teasers from the director: the idea and the look. And the first still shots from the film.

This kind of has why am I still programming written all over it for me.

Podcasts are more popular than ever. We’re lucky right now to have a bunch of podcast networks and great iOS clients, including the newly-released official Podcasts app from Apple. My favorite remains Instacast on iPhone, but there are other good choices like Downcast.

It’s never easy for developers when Apple arrives into your market with free competition, especially if it might one day be bundled on the OS alongside the Music and Videos apps. I wish the third-party guys the best of luck.

But for podcast creators, the extra exposure can only be a good thing. I hope we can welcome even more listeners to our Core Intuition podcast. We just opened a new way to send in feedback and questions, too: Glassboard. Use invite code COREINT on the web or iPhone app to join the board and get a little behind-the-scenes look into the podcast.

Chad Sellers has a post comparing Mac App Store sandboxing to mistakes from Linux, with this very reasonable advice:

"I believe that Apple should have at least led the way by sandboxing all of their own apps sold through the Mac App Store (I believe they have not sandboxed a single one of their 17)."

This reminds me of Twitter. When Twitter forced third-party clients to move to OAuth, but didn’t change their own app to use it, many developers said it was a double standard. Twitter’s response: the official Twitter app was part of the service, not really a separate app, so it didn’t need to use OAuth.

Maybe Apple could make the same case for Mac OS X’s built-in apps: Address Book, iCal, and Mail don’t need to be sandboxed because they are part of the operating system. But that argument doesn’t work for Keynote or iMovie. Those apps should play by the same rules that all productivity and video software in the store does.

If Apple were to sandbox a few of these it would go a long way toward convincing developers to do the same. And it would also shake out bugs and missing APIs in the whole sandbox environment.

David Barnard shares the story of App Cubby’s Timer app, along with this story of a failed project that got out of control:

“Back in the Spring of 2009 I had this great idea for an interactive music app. A series of 2D/3D scenes that would react in real time to music as well as user input. That idea became Project VJ, a project I worked on for the better part of a year. After hundreds of hours of my time and $50k in expenses, we had a decent prototype. It was a gut-wrenching decision, but in April 2010 I shelved the project, realizing that it would take at least another year and a lot more money to ship.”

This must have been very disappointing. It’s bad enough for a side project to fail, something that you’ve only invested your time into. With the kind of investment that David talks about it is surely even more difficult to let go.

I used to get this wrong – too many apps, not enough polish. There are a few products I worked on that never saw the light of day. But I don’t do that anymore. Everything I have worked on in the last 2 years has shipped, in one form or another.

You have to ship everything because you never know what is going to hit. David Barnard and Justin Youens’ latest, Launch Center Pro, which started as a few-week experiment, reached the iPhone Top 10. You have to ship everything because time is precious. Make the decision early on about whether to start. If it’s worth coding, it’s worth letting the world see.

TextExpander 4 shipped this week, and with the update it breaks from the Mac App Store and instead requires customers to buy directly. TextExpander is the first popular app I’ve seen to do it.

Moom is another one that actively encourages users to move away from the store. Recently on launch Moom displayed a news window that included this:

"Apple has activated sandboxing on the Mac App Store; under the sandboxing rules, we can no longer add new features to the App Store version of Moom (we can only fix bugs). However, we have a method by which you can migrate (at no cost) to our direct sales version of Moom, which has no such limitations. For details on how sandboxing affects our apps, and how to migrate to the direct sales version, please read this article on our blog."

Even Panic – frustrated with the long approval times for Coda 2.0.1 – is experimenting with how best to let Mac App Store customers migrate to the direct version. See this tweet and screenshot from Cabel Sasser.

This has been a theme on the last couple episodes of Core Intuition. Daniel Jalkut and I talked about how we feel about sandboxing after WWDC, and more on my decision to migrate Clipstart out of the store. Things are getting better in Mountain Lion, and I’ll revisit my decision next year, but for now I think I made the right call to focus on work outside the Mac App Store.

(And if you haven’t listened to the podcast recently, check out the new episodes and subscribe. We’re now a weekly podcast!)

From a great story on Mark Zuckerberg in New York Magazine:

“‘Mark has done two things in his twenties,’ a colleague of Zuckerberg says. ‘He has built a global company, and he has grown up.’ The second one made the first possible. When early mistakes risked an employee mutiny, Zuckerberg knuckled down and learned how to lead.”

It covers founding the company, Sean Parker, other execs, the Instagram purchase, and more. I’ve got new respect for what Zuckerberg has accomplished after reading this.

Several years ago, when Yahoo or Microsoft or whomever was rumored to have offered a billion dollars to buy Facebook, I thought Zuckerberg was foolish and arrogant for not selling the company. I didn’t get how it could be a real business, ever worth more. Looks like I was wrong.

Of course, the narrative of the last week is that the IPO disappointed, with the stock steadily falling since its debut. But that’s only relevant if you want a quick win. The long-term view says that Facebook is here to stay, and that the company’s growth is a reason to celebrate. Not for the service or founders, but for the success of so many great developers and designers who are now part of the public company: Instagram, Gowalla, Push Pop Press, Sofa, and others who are working inside Facebook, trying to build something great.

Just approved in the Mac App Store, Clipstart 1.4.2 fixes upload issues with YouTube and especially Vimeo, which was broken in previous releases because of Vimeo API changes. I expect this to be the final Mac App Store release for Clipstart. As I blogged about before, all Mac App Store customers can upgrade to the direct download version for free.

Here’s what you should do if you bought Clipstart from the Mac App Store:

• Update to Clipstart 1.4.2 using the App Store app.

• Run Clipstart at least once, then quit it. This allows it to copy the Mac App Store receipt so that it's accessible to other versions of the app.

• Download the latest version from the main Clipstart web site and replace the existing copy in your Applications folder with the direct download.

And if you're thinking about buying Clipstart for the first time, please get it directly from the web site and checkout via PayPal. There's also a demo with support for 2 uploads and tagging 20 videos.

I’m now turning my attention to version 1.5, which will improve a few things and add support for Gatekeeper on Mountain Lion.

I’ve been using Twitter’s Bootstrap in an internal project at VitalSource for a few months, and over the weekend I finally switched to using the CSS framework in Tweet Marker too. The layout now works in more browsers and provides a much better foundation for design changes. It also allowed me to integrate this excellent date picker.

Here’s a short screencast video showing the date picker in a new browsing feature in Tweet Marker Plus. I’m very happy with how this turned out — both the look and functionality. On the server the date ranges are implemented with a Sphinx query, so they can be combined with search terms to help find old tweets.

A few things have changed since I last talked about hosting. Tweet Marker passed 200,000 total users to the API. There are more apps and more platforms. And of course Plus launched with new requirements for database and search indexing of tweets.

Here’s a graph showing monthly hosting costs for the last year, stopping just short of $600/month for April:

The first dip was when I moved away from Heroku’s dedicated PostgreSQL database, to Redis on EC2. The more recent increase is when I updated capacity for Tweet Marker Plus.

Tweet Marker currently runs on both Heroku and Amazon EC2. On Heroku, there are 6 dynos: 5 web dynos with 3 Unicorn processes each, and 1 background worker. I also run hourly scheduled tasks that add a small number of extra dyno-hours, and sometimes I’ll fire up additional temporary workers.

For Amazon, I run with 3 medium server instances: 1 for Redis master, 1 for Redis slave, and 1 for search with MySQL and Sphinx. The search server is partitioned across multiple EBS volumes, each one mounted as a separate MySQL database and Sphinx index. It is possible for me to move a database or search index shard to a different EC2 instance if I need to, as well as move customers between shards.

The volumes look like this in Amazon’s admin UI:

I picked 20 GB shards because it seemed like about the point where the database would be too big to be fast, given the modest hardware. It’s enough to hold several months of tweets for all the users in a shard. I estimate how many users should be in each shard, and when it reaches that number I roll new accounts to the next shard, and so on.

Backup dumps for the Redis database and MySQL get sent to S3 every hour and every day, where I keep the last 24 hourly backups and the last 31 daily backups. I also do occasional EBS snapshots.

I don’t currently need a MySQL slave for backups. If I lose a drive, when I restore the last hourly backup, Tweet Marker Plus will automatically add any missing tweets lost during the downtime to bring things back to a current state.

Overall I’m happy with this setup. It’s as simple as I could design it. Hosting is not cheap, but I think I can run for the rest of the year with very few changes and mostly fixed costs. I also plan to switch to reserved instance pricing at Amazon, which should be a significant discount.

If you’ve made it this far, you probably care enough about servers and Twitter that you should consider signing up for Tweet Marker Plus yourself. Check out the details here.

“Around here, however, we don’t look backwards for very long. We keep moving forward, opening up new doors and doing new things, because we’re curious… and curiosity keeps leading us down new paths.” — Walt Disney

There are many posts on this blog about Wii Transfer, the little Mac app that launched commercially almost by accident, and convinced me that it would be worthwhile to invest time in this side business called Riverfold Software. Early posts like the launch post in 2006 or this one about the first 75 days, and this one covering the price bump in version 2.5. But the app has been fading over the last couple of years, no longer as relevant today as it once was. It’s time to let it go.

I’m retiring Wii Transfer to focus on my other apps. It’s not that it doesn’t sell; it still does. It’s just that it’s not an app I actually use anymore. By officially shelving the whole project, I hope to remove a psychological burden of sorts — to no longer worry that I’m ignoring an active product.

It also doesn’t fit into a new theme I have for Riverfold: apps that are all about keeping and remembering what matters. For Clipstart, that’s family videos. For Tweet Library and Tweet Marker Plus, that’s old tweets. Wii Transfer is about… listening to music on your Wii? It doesn’t fit, and in the world of the Apple TV and Roku, modern streaming technology has passed the app by.

If anyone is disappointed that Wii Transfer will no longer receive updates, of course I offer refunds. I won’t be selling or open sourcing the app, preferring instead to continue to support existing customers myself for as long as they want to use the app. And I’ll keep the automatic bookmark service running that makes setup easy, as well as the Mii rendering service, so nothing breaks.

I put a lot of work into Wii Transfer over its 5-year lifespan. It’s not easy saying goodbye, especially to some of the unique things that only Wii Transfer could do, such as exporting Miis as images. Maybe I can bring that back one day. For now, I’m following the path started by my apps Tweet Library and Clipstart, for which there are many new things still to do.

I released a small bug fix update to Clipstart today, version 1.4.2, to fix an issue with YouTube uploads when using your Gmail address sign-in instead of the YouTube account username. This version should also show up in the Mac App Store after it goes through Apple’s approval process. You can see the full release notes for recent bug fixes here.

As I said earlier this year, there will only be a couple more releases of Clipstart in the Mac App Store. My current plan is to switch completely over to direct-only sales with version 1.5. The new versions run without prompting for registration if you’ve already purchased and run a copy from the Mac App Store.

You may not notice it right away, but Tweet Marker Plus is the start of a big migration for my Twitter projects. The backend infrastructure for tweetlibrary.com will be moving there, so that Tweet Marker can have access to published collections. And a major web feature to complement Tweet Library — originally written for tweetlibrary.com last year but never released — will be launching on Tweet Marker Plus instead.

Essentially, Tweet Marker will be the web app and web services. Tweet Library will be the native iOS client only.

This has a few pretty big advantages for me:

• One web app to manage instead of two.

• Other Twitter apps can access the collections publishing API.

• Tweet Marker is a better brand than Tweet Library.

More people know about Tweet Marker. It just makes sense to build any new web features on top of that existing name.

The first step in this transition is ready now: published collections can be accessed via tweetmarker.net. For example, see this collection of tweets from the game Millinaut by Shaun Inman, Neven Mrgan, and Alex Ogle.

VoodooPad 5 is out this weekend, with a new file format that plays nicely with Dropbox. Gus Mueller says:

“No more connecting to WebDAV servers and having to deal with authentication issues and strange HTTP errors. Now you can just put your VoodooPad 5 document in a Dropbox folder and VP will detect when pages have been updated.”

This is another good example of where web APIs like Dropbox can be more useful than iCloud. Multiple people can collaborate on a VoodooPad document, and the direct download and Mac App Store copies of VoodooPad can sync together.

Also new in version 5 is native support for Markdown and an ePub export option. The workflow for help documentation that I posted 5 years ago carries over to VoodooPad 5 just fine, too.

There has been some good commentary on Sergey Brin’s interview with the Guardian. It’s probably best summarized in John Gruber’s comment:

The assumption here is that the only way to search is through Google, and that the "open Internet" is only what Google can index and sell ads against.

This resonates with me because I think Google has put enough ads on the internet. It’s impossible to take anything Google says at face value if they talk of “open” but their intentions say “ads”. But here’s the thing: Brin is actually right. There is great data hidden behind apps that should be indexable.

In the race to win the App Store, we forgot about the web. Think about Instagram. Millions of photos are being shared that are inaccessible via a search engine. These photos can’t be found again and aren’t discoverable. When you search the web, how does it make sense that public photos on Flickr show up, but public photos on Instagram do not?

We shouldn’t have to choose between Apple’s closed systems and Google’s ad-driven business. I want to talk about improving the web without automatically being pro-Google. This tweet from John Marstall made me realize it:

I'd prefer to separate "indexable" from "indexable by Google." Yes, they're mostly the same for now. Might not always be.

We need competition in web search. More than Bing. A new search engine is the number 1 item from Paul Graham’s frighteningly ambitious ideas:

The best ideas are just on the right side of impossible. I don't know if this one is possible, but there are signs it might be. Making a new search engine means competing with Google, and recently I've noticed some cracks in their fortress.

Of course it’s crazy, but so was the 1st search breakthrough, lightning-fast AltaVista, and so was the 2nd major innovation, Google itself. It’s time for a search engine that isn’t all about ads. It’s time for search that understands apps and embraces data from web services as much as it does from web sites. It’s time for the 3rd act in search.

“This MacStories article”:www.macstories.net/stories/i… covers the progress that developers have made adopting iCloud over the last 6 months. Over the next 6 months, we should have an even better appreciation for what iCloud is and isn’t good for.

For iOS backups and iTunes Match, iCloud is fantastic. For private, app-specific data that doesn’t make any sense away from a single developer’s native Mac and iOS apps, it’s also excellent. There’s no question that using Macs, iPhones, and iPads today is a significantly better experience thanks to iCloud.

But there are two fundamental limitations in iCloud that make it inappropriate for a bunch of syncing uses:

• No way to access it from other platforms or web apps.

• No way to share data between apps from different developers.

I don't expect Apple to change this. Again, for what iCloud was made for, these limitations are fine. It simply means that iCloud cannot replace web APIs that do solve these two problems.

“Here’s what I wrote”:www.manton.org/2011/06/f… not long after announcing the Tweet Marker API:

"The primary goal with Tweet Marker is to enable different Twitter apps to work together. iCloud is designed for storage and syncing only between apps from the same developer, so it's not appropriate as a replacement architecture for Tweet Marker."

Think back to the so-called Web 2.0, which gave us web services to access previously closed-off data. This eventually led to truly dominant syncing APIs like Dropbox, Simplenote, and Instapaper. At the same time, we all have more devices than ever before. Syncing exploded on iOS in text editors and note taking apps. Without a proper shared file system between apps, or even an event or scripting system, these iOS apps looked to web APIs as the only way to communicate.

That has turned out wonderfully. With web APIs as the only solution, we see more compatibility between apps and more web services popping up all the time. If you create a new web app, it’s dead without an API. Every success of the modern web, from Flickr to Twitter, has an API that is available from any platform.

So then what about iCloud? If Web 2.0 made data more accessible, iCloud takes that same data and… keeps it closed. It’s a step forward on user convenience and a step back on interoperability.

If you’re a developer considering iCloud support, just make sure your data fits there. Ask yourself if your data is all about your app, or if it’s bigger than your app. Developers who are willing to take a risk on building an open API instead of iCloud could see new opportunities: web-based views of their data, compatibility with other apps, and syncing on the Mac outside of the App Store.

A couple years ago, “Shawn Blanc said about cloud syncing”:shawnblanc.net/2010/08/s…

"And my next wish? A cloud-based service like Instapaper, but for to-do items. I want it to be available in apps like Tweetie, Reeder, and more, so when I click on 'Do Later' it sends the link or item of note into a running to-do list (that syncs with Things, of course)."

Imagine if Things or OmniFocus or another tasks app opened up a slice of their private syncing API to make the Instapaper of to-do inboxes. Now take other APIs for all of the useful apps we use. Not just to-do apps as Shawn mentions, but RSS, photos, blog drafts, sketches, and more.

What I wrote above about Tweet Marker is still very much true. Since then, Tweet Marker has become the standard for last-read syncing between Twitter apps, with support for 15 apps and many more in development. I want to see more syncing platforms like this. Let’s think big and make apps work together.

When I created “Tweet Marker Plus”:tweetmarker.net/plus, I thought I was creating a new way to search Twitter. Limit the search to just people you follow and you can store more tweets, and more relevant ones. But as I’ve been adding new features to it, I’m realizing that Tweet Marker Plus is really a new kind of Twitter client — a client that has search and filters at its core.

Here’s what the sidebar looks like in my Tweet Marker Plus account:

Seems simple enough. But quickly switching between saved filters is very powerful. Because Tweet Marker is routinely fetching new tweets in the background, even when you haven’t opened your web browser in days or weeks, there are no gaps in the timeline. When I use a filter, it’s showing me everything that any of the people I follow have said since I first started using Tweet Marker Plus.

I’m excited about this. I’ll keep adding features and growing the storage, to make Tweet Marker Plus the best value $2/month could possibly get you.

I love reading about how big sites use Amazon EC2. If “this post from Instagram”:instagram-engineering.tumblr.com/post/1364… is still accurate, they must be at something like $50k/month in hosting costs. Their user base has doubled in the 4 months since they posted that.

My setup for Tweet Marker is trivial by comparison, but to me — without Instagram’s $7 million in funding — it’s a very big deal. What I wrote “back in October”:www.manton.org/2011/10/r… about moving to Redis is still mostly true, although I’ve added MySQL and Sphinx to the mix. I now run with 3 Amazon EC2 “medium” instances and have 5 web dynos (with 3 Unicorn processes each) at Heroku.

I put everything from donations back into the servers. For Tweet Marker to work, it had to be rock solid. It had to scale. It’s only the first week for “Tweet Marker Plus”:tweetmarker.net/plus subscriptions, but already I have a good feeling that it’ll all pay off.

Tweet Marker is going really well. It’s growing fast, users love it, and it has wide support in all of my favorite Twitter apps.

Today I’m announcing the next step for the service: Tweet Marker Plus. This is a paid subscription with additional features, such as a brand new search and a web-based timeline that syncs with Tweet Marker. Along with Plus, I’m rolling out version 2 of the API.

We’ve learned a lot over the last few months. Here are some of the things that I wanted to improve for the next version of Tweet Marker, both for the API and the business.

• Consistent behavior across clients. It was fine at first for everyone to experiment with their own way to use the API, but now it's time to come up with some best practices to guide developers.

• More frequent sync. Mac and iPad clients in particular — where the app may stay in the front for some time — need to be hitting the server at regular intervals to catch changes from other devices. I believe this is the most common source of problems.

• JSON-based API. The "v2" API should use JSON and carry more metadata, such as timestamps for the tweets and when it synced.

• Profitable hosting. Although I've accepted donations since nearly the beginning, hosting costs are way up. I didn't originally intend to turn this into its own business, but that is the clear way forward. It needs to bring in consistent revenue to cover hosting now and into the future.

With paid subscriptions, I can tackle the above bullets and also add more features. The first new feature to launch as part of Plus is a new search. Tweet Marker Plus indexes tweets from your friends so that you can find tweets later. It's the solution to use when you don't want to search all of Twitter, but you also want a large collection of tweets, not just the most recent ones that many clients store and filter.

Tweet Marker Plus is $2/month. You can sign up today at “tweetmarker.net/plus”:tweetmarker.net/plus. Also check out “the screencast”:www.riverfold.com/software/…

The API documentation has moved to Github and is “also available now”:github.com/manton/tw…

Every year on March 9th, as SXSW is getting started, I like to mark the anniversary of this blog. This time it’s the 10th year.

My second post back in 2002 was about a panel run by 37signals. I wrote:

Ernest and Jason really get it – I hope they inspire some designers to think about web sites in a new way, and finally start focusing on usability and page load time and cut the fancy graphics, roll-overs, and animations.

This was a couple years before they reinvented themselves as a software company with Basecamp. As the new Basecamp launches this week, it’s fascinating to think back on how far 37signals has come. The web is bigger now and more complex. Subscription web apps are everywhere. But I think the focus on performance that drove Jason Fried and his original co-founders to promote simple design in that SXSW panel a decade ago is still very much at the heart of what 37signals does.

The day after I “wrote about removing Clipstart from the Mac App Store”:www.manton.org/2012/02/s… Apple announced that the sandboxing requirement would be delayed again. In that announcement was also a new twist: sandboxing would not be required for bug fix updates to existing apps.

This is welcome news, but I stand by my post. I still plan to transition Clipstart away from the MAS. The difference now is that I can do it at my own pace, providing a new version or two to MAS customers that will make the move easier.

I’ve already gotten started. Clipstart 1.4 just shipped with a few new features and better support for recognizing MAS receipt files. I’ve also submitted it to the Mac App Store, where it is waiting for review.

It’s not clear where we are going to end up with sandboxing. “Quoted in Macworld’s coverage”:www.macworld.com/article/1… Paul Kafasis suggests that sandboxing is so flawed that Apple should just scrap the whole thing.

“Michael Tsai talks about”:mjtsai.com/blog/2012… all the work that is required to stay in the store. He closes with something that I’ve been thinking about:

"At each step of the way, it looks like just a little more work to get into the Mac App Store, or to stay there. Until the next issue pops up. And then, if you're successful, you're sort of locked into it due to the reasonable expectations of your many customers."

This lock-in creates two immediate problems with leaving the Mac App Store:

• What about customers who may have originally bought the app directly from me, but decided to "upgrade" to the Mac App Store version? I'm sure I only have a few of these, but it's still unfortunate if someone bought the app twice. I'm just glad I didn't follow the lead of MAS-exclusive developers, such as "Pixelmator's effort to encourage customers":[www.pixelmator.com/blog/2011...](http://www.pixelmator.com/blog/2011/01/06/transition-to-the-mac-app-store/) to switch to the MAS.

• How do current Mac App Store customers get notified that there is a new version available outside the store? Sparkle is not allowed in the MAS, but I think the right thing to do here is provide a similar notification in the app. It would link to a web page with instructions for downloading the app directly.

I'll admit I have some regret leaving the Mac App Store. It's just so convenient for purchasing and installation. If I'm going to make this work, I'll have to redesign my own rather clunky purchase and activation experience. And I'll have to do a much better job of marketing, something that has not been easy with Clipstart.

I wrote a draft of this post a few weeks ago, before Mac OS X Mountain Lion was announced. It was pretty critical of Apple’s aggressive approach to sandboxing, and I’ve kept most of that, but the new Gatekeeper feature for Mountain Lion at least gives me a way out. I don’t think Apple would have created Gatekeeper if they planned to abandon apps sold outside of the Mac App Store.

For the next release of my app Clipstart, I will be removing it from the Mac App Store and only selling directly from my web site. With Gatekeeper I hope to have some confidence that my customers will still be able to run the app on future versions of the OS.

But let’s take a step back, to a good blog post from Craig Hockenberry on moving xScope to use sandboxing. One of the most important things to keep in mind is that what works for one app may be unsuitable for another. Craig touches on this with an example from Panic’s Transmit:

"Of course there are some applications that have a harder time than others: primarily if those apps require access to all or part of the filesystem (think about syncing data with Transmit, for example.)"

Clipstart also falls into the same “needs to access the whole file system” category as Transmit. It’s not just one feature; the whole app is based on the fact that it can point to video files anywhere on the system, or manage your video library in a central location on any hard drive. These are things that are difficult to do in the sandbox, but even worse, I don’t see a clear path forward for existing customers to move into such a restrictive environment.

Maybe I could file bugs with Apple for exemptions, and reduce the functionality of my app to fit within the limits of the sandbox, but I’ve made the decision that it is just not worth it. I would much rather spend 100% of the time I have for Clipstart on new features only, not playing catch-up with Apple.

Atlassian has made a similar decision for their app SourceTree. On the sandboxing restrictions:

"The trouble is, the sandboxing implementation currently in place on Mac OS X Lion doesn't allow for all the behaviours that real Mac applications do _right now_, behaviours which are not at all contentious, are approved in the Mac App Store already, and indeed are very much appreciated by users."

Daniel Jalkut continues this argument, saying that sandboxing could be good for developers, if only the current entitlements weren’t so very incomplete. That’s true. But we can only make decisions based on what entitlements and APIs are there today, and today it’s not enough.

I will try to make this casualty of sandboxing as painless as possible for Clipstart customers. I will honor Mac App Store receipt files so that everyone can migrate to new versions of the app. And I’ll provide extra serial numbers to anyone who asks, for fresh installs on machines that never had the Mac App Store version.

Clipstart has turned out to not be a very good fit for the Mac App Store anyway. It’s the kind of app that you need to download and try out before committing your whole video library to it. Sandboxing is just the latest and most significant in a series of frustrations with the MAS.

For my customers, sandboxing isn’t actually a feature; it’s a bottleneck to getting work done. I can’t justify spending any time on it. I already have a product and platform (Tweet Library for iOS) where I can play the app review game. I want my Mac app to be a break from that, with a total focus on making the app better and a release schedule and feature set that I control.

So it was a relief to hear about Gatekeeper. I don’t want pulling Clipstart from the MAS to automatically doom the product, and now I don’t think it will. Instead of shying away from features that won’t work in the sandbox, I can even embrace them as a competitive advantage. I’m more excited than ever to get back to Mac development without this decision and chilling effect hanging over my head.