Tweet Marker wins a Macworld Eddy
A few times since it launched, I've said to friends that Tweet Marker may be the best thing I've done. It has reached more users than any of my indie Mac and iOS apps, and it has been especially rewarding to work with other Twitter developers. It's not perfect yet — there's more to improve in future versions of the API and clients — but I smile every time I see a tweet about how someone can't imagine going back to a Twitter client without it.
So it was really gratifying to see Macworld recognize Tweet Marker with an Editors' Choice Award for 2011. Thank you Macworld for seeing Tweet Marker as an important part of the Twitter experience.
And thanks to all the Twitter app developers who have supported Tweet Marker in their apps. We are up to 9 supporting apps across 5 platforms — Mac, iPhone, iPad, Chrome, and Android — with more on the way. I've opened up the API to over 40 different clients in various stages of research or testing.
Tweet Marker is a little unique among most of the other Eddy winners this year in that it's still completely free. I won't see a sales spike following the announcement. If you've been enjoying the service, consider picking up a copy of my iPad app Tweet Library, or donating directly on the site.
December 8, 2011 02:38 PM [link]
Redis-only migration
The best critique and praise for Heroku is that it's opinionated: only deploy with Git, only use PostgreSQL. On the whole this is a good thing, because it simplifies the choices that developers have to make. Fewer choices means more time to spend on things that matter.
But at $200/month for a dedicated database, there's a very real cost with PostgreSQL on Heroku. For a service like Tweet Marker that is currently free, I wanted to save money on hosting and simplify things down to a single database. I might have preferred to streamline to PostgreSQL-only, since it's rock solid and Heroku's tools are excellent for managing backups and moving data around, but Resque (which Tweet Marker uses to process background work) is built on Redis. If I wanted to streamline to a single database, it had to be to Redis.
Tweet Marker as essentially an OAuth-wrapped key/value store anyway, so why burden the app with a full relational database? The handful of extra tables I had for tracking users and sessions could also be migrated to Redis. And even though it's a memory database at heart, it is persistent, so I can use it for real data.
I conducted the transition in stages:
- First, I was using Redis for stats. Because Redis has an atomic increment command, it's trivial to use it to keep counters on various metrics: total number of reads/writes per hour, day, or month, broken down by different Twitter clients. This was my first use of Redis outside Resque tasks and it performed great.
- Next, I moved the marker data. I wrote a script to dump all the data from PostgreSQL to Redis, and at the same time modified Tweet Marker to write data to both locations when requests came in. I let that run for about 2 days to confirm that the data was identical in both places before shutting PostgreSQL down and routing reads to Redis only.
- Finally, I finished moving some additional tables to Redis so that I could completely shutdown PostgreSQL. Again, simple is better.
All of this was using Heroku's Redis To Go add-on. I love it because there's no configuration; click a couple buttons and you have a server. But as soon as I was running full time on Redis To Go, I started to be concerned that I had given up a great amount of control over my data. While Redis To Go conveniently offers daily backup snapshots, it doesn't appear to have any commands for restoring data or otherwise getting access to the server and Redis files. And though I could probably set up a separate Redis To Go instance and manually send it a slaveof command to turn it into a slave, not being able to edit the config file meant that if the server was rebooted I would lose the slave until it could be reset. Not good.
I also expected I would quickly outgrow the smaller Redis To Go plans, eventually forcing the monthly pricing up to the same level as I was paying for PostgreSQL, but with less control. I had to run my own database server instead. And because Heroku is all on Amazon EC2, it also had to be on EC2 for the best performance between web server and database.
I went to work provisioning an EC2 instance. Initially it ran as a Redis slave to the master on Redis To Go. After I felt comfortable with EC2 and had automated backups in place, I added a separate instance on Amazon for the master and switched the app over to it. The cost for 2 "micro" instances is about the same as I was paying for Redis To Go, but with much more available memory, hourly backups to S3, and access to the Redis aof log. And my total monthly hosting costs are about half what they were before.
I'm hoping to keep this setup for at least the rest of the year and well into next. As the Tweet Marker user base grows, I'll bump up to higher-capacity EC2 instances. Heroku continues to be really great, so I don't expect to change anything there.
October 25, 2011 03:20 PM [link]
SXSW 2012 and Twitter API innovation
The short version of this post is: please vote for my SXSW talk, which I've proposed with David Barnard of App Cubby. If selected, we'll be talking about how to innovate on top of Twitter, using examples from the history of Twitter apps in the App Store including Tweet Marker, Tweet Library, and David's upcoming app Tweet Speaker. Equal parts business and APIs, I hope it'll capture how much we can still do if we think beyond the Twitter basics.
The longer story is that SXSW is always changing. I started this blog on the first day of SXSW 2002, when the conference was just a few rooms in a single hallway, and I've seen it grow to more than a few venues spread across downtown Austin and even farther out. That's okay. There's a place for small events, as SXSW once was, and there's a place for the event where the blogging, design, social network, and software folks can meet in one place.
Take the Mac and iOS development world. This year alone has NSConference, Voices That Matter, 360|iDev, 360|MacDev, CocoaConf, MacTech, Çingleton, and SecondConf. These attract developers from all over the country, but most people can attend one at most, and many events are regional conferences at heart.
We need WWDC as the single place. No matter how great the smaller conferences are, WWDC is the big one, the one you don't miss. And so it is with SXSW.
I believe SXSW 2012 is going to be fascinating to watch. A couple years past when everyone already thought it was too big, 2012 could see real turnover. Some previous attendees will skip it, and many new people will speak for the first time. I want to see that conference, to find out what its themes and focus will be, and hopefully our talk can be part of it.
September 1, 2011 10:06 AM [link]
Tweetbot + Twitterrific
The second app to support Tweet Marker has arrived, and it's a great one: Tweetbot. I love using Tweetbot on the iPhone and Twitterrific on the Mac and iPad. Seeing these apps work together just makes for a better Twitter experience.
Here are a few tips to keep in mind while using apps that support Tweet Marker:
Turn it on! First things first: enable Tweet Marker in both apps on every device you plan to use Twitter on. MacStories has a good summary of this (with screenshots) for their coverage of Twitterrific and again for Tweetbot.
What syncs? Only the main timeline and mentions sync between the apps. While working with these developers, we made a decision to change how Twitter lists sync after Twitterrific had already shipped. So while both apps think they are syncing lists, the format on the server is not compatible between the two. A future version should fix this.
How to switch apps? Most apps take the approach of saving the last-read position when the app is quit or sent to the background, and restoring it again when it is launched or brought to the front. You may need to change your default habits a little to get the most accurate behavior. When you know you are going to switch apps, just close your active Twitter client first before opening the next one. It may also help to wait for the app to download new tweets before scrolling or interacting with it.
August 28, 2011 10:41 AM [link]
Hosting for Tweet Marker
Today was day 7 of the first app to ship with Tweet Marker support. Overall, things have been working great. I thought I'd write a little bit about my hosting experience so far.
First, the good news: the service is up and fast. Twitterrific is sending a lot of traffic, even with the preference off by default, but the server app is architected in such a way that things are zipping along nicely. Performance is especially important on the iPhone, where a Twitter client might be saving state on quit and any lag would be pretty obvious. Non-trivial server work, such as hitting the Twitter API, is done in the background using Resque, and the queue size there is usually small too.
There have been two distinct problems, though. Each was a good opportunity to improve the system.
On the first day, the background queue stalled and the requests piled up, so much so that my Redis instance ran out of memory and clients started receiving errors. I upgraded Redis, restarted the app, and bumped up the number of processes to quickly catch up on the backlog. Although it hasn't resulted in errors since, it has delayed some requests on 2 more occasions after launch. I also adjusted how I'm collecting statistics to further improve performance.
More recently, I've noticed some sporadic SSL errors when Tweet Marker tries to verify the user account via Twitter. I added more wait-and-retry logic, and I'm keeping an eye on this to kick the background script if necessary. There have been no errors in the last couple days.
I still think it was the right decision to host on Heroku. Although I am manually monitoring the server for general health (need to automate this badly), everything is less work than I would do with a colocated box or VPS. The biggest cost is that at the last minute I upgraded to Heroku's dedicated PostgreSQL database. I'm still evaluating that, but I would rather launch with too much horsepower and scale it down later, than not enough.
Again, thanks for your support. It is a little nerve-racking because I know that other people are depending on Tweet Marker being up and performing well. This is a different experience for me than working at a larger company with a dedicated system administrator, and also different than server backends for my Riverfold products, where it is only my own customers who will be disappointed if I fail.
August 15, 2011 10:06 PM [link]
Wii Transfer 2.7.2
And now for something completely different. I released a bug fix update to Wii Transfer last night, the first in over a year. It doesn't have some of the bigger things I'd like to finish for the app, but it does have an important bug fix to Mii rendering.
At some point in a recent update to Safari or Mac OS X, Miis started appearing blank in Wii Transfer. These are rendered in an offscreen window using Flash, then saved as thumbnails in Wii Transfer with the option to export a JPEG. The fix — for reasons I'm still not clear on — was to switch to using NSBitmapImageRep's cacheDisplayInRect instead of initWithFocusedViewRect. Needless to say, customers were frustrated that this was no longer working, and I'm sorry I didn't take care of the problem more quickly.
I've been thinking a lot lately about the future of Wii Transfer and its companion @wii Twitter account. This little Mac app is overdue for a rename and refocused feature set.
August 14, 2011 05:29 PM [link]
Tweet Marker bootstrap
Tweet Marker is getting big. I couldn't be more excited that Twitterrific 4.3 is now shipping with syncing across Mac, iPad, and iPhone. Users love it and many are rediscovering Twitterrific or becoming new customers just because of this feature.
More apps are coming, too. Craig Hockenberry, from a Macworld article by Dan Moren:
"The real value is having something that works well across the entire Twitter ecosystem. It's our belief that Tweet Marker will become more useful as more clients support it."
I struggled with how to build, price, and launch Tweet Marker. The first developers to use it are taking a risk, so I felt I had to remove the friction of supporting the service by making the API free. To cover hosting costs, I plan to later make an optional paid subscription available for developers who want more than the basics.
But we're not there yet, and some people have asked how they can support Tweet Marker directly. As an experiment, I've put up a donation button over on the Tweet Marker site. Think of it as a small investment in the service, a bootstrap to get things off the ground and remove the stress of scaling.
It is not, however, a substitute for supporting client developers. Please pick up a copy of Twitterrific, and when my app Tweet Library is available with sync I hope you'll consider that as well. Thanks!
August 13, 2011 01:23 PM [link]
Twitter app chance
Five years ago today, I joined Twitter as its 897th user, though for some reason my first tweet wasn't until a few months later. So much has changed in the meantime — the API always in flux, the transition from primarily SMS, to web, to apps — but in many ways the core of the service remains intact and stronger than ever. Short messages, distributed efficiently to friends.
I talked about some of the good and bad of being a Twitter developer on the ATX Web Show last week. There have been a string of changes that cause developers to scramble: turning off basic auth, discouraging mainstream clients, disabling DMs for xAuth. With each step, Twitter loses a little goodwill, and that's demonstrated in the tweets I collected over the xAuth change.
Even as Twitter passes 1 million registered apps, there's a risk that some developers will stick with the platform as users only, putting their apps in maintenance mode. In May, Kiwi developer Isaiah stopped development of his Mac Twitter app:
"I'm just going to take a break from Kiwi for a while. It's still for sale. I still support it. I'll still fix bugs when they crop up. But adding new features and playing catch up with the other guys/gals is off the table."
Maybe because I don't have to depend on Tweet Library sales, I tend to more stubbornly ignore what is good business sense. There's so much I still want to do. As I wrote in my previous take on the state of the platform: "I'm a little discouraged, but not enough to stop."
I think that's doubly true today. More annoyed, but also more determined to plug holes in the platform, from archiving to syncing. I couldn't be more excited about the developers who are building in Tweet Marker support.
And there's always a chance, a feeling that something big is just around the corner. That if I don't add that one feature, or open up that new API, I'll miss the tipping point that makes Tweet Library really take off.
July 14, 2011 11:45 AM [link]
iPad Pro is the new iPod Photo
I'm fascinated with the iPad "3" rumors because on the surface they make so little sense. Apple just shipped the iPad 2, no competitors can match it, and demand is strong. Why mess with a good thing so soon?
But it almost fits when you give it a name like "Pro" (or iPad Retina, or whatever). This isn't a replacement for the current iPad; it's another layer to the product lineup. And like the awkwardly-named iPod Photo from 2004, I bet the iPad Pro is meant to be temporary. It's a way to sell a high-end, over-priced and over-pixeled iPad before the technology is cheap enough for the masses. A year or two from now, the Retina Display will be available in all iPads, and the "Pro" name will fade away, just like iPod Photo did when all iPods got a color screen.
July 12, 2011 06:00 PM [link]
Push-based sync
Guy English writes about iCloud and the magic glue (Push Notifications' persistent connection) that makes it work:
"Each of these new features tickle the persistent 'push' connection and trigger some action on the device. The short-form state may be transmitted immediately and set on any connected device within moments. Document syncing is likely to trigger a negotiation process to compare the state on any one device with The Truth stored on Apple servers and replace the document on the device with the latest revision — this has the advantage of limiting the window between syncing where conflicts are most likely to occur."
Sync speed matters. The first note sharing server I built for VitalSource years ago assumed a lot of offline time, and despite my blogging in 2007 that it was "magic", in practice it could take 5-10 minutes before all your computers got their act together to get a set of highlights completely synced. With that kind of lag, note edits might happen on a client in the meantime, so we remembered conflicts everywhere and had a UI for resolving them.
Too complicated. The new system, recently rolled out in Bookshelf for iPhone and iPad, syncs so much more efficiently and quickly that conflicts don't need the same emphasis. We can throw away a bunch of code and simplify the user interface.
I've yet to do anything with iCloud except read the release notes and sit through a couple WWDC sessions, but we're going to have a fantastic platform if it can deliver the same speed and reliability of Push Notifications. Guy's post is the first I've seen to connect the dots, capturing how well-positioned Apple is to use this plumbing for all sorts of stuff.
July 1, 2011 11:13 AM [link]
Direct download as a bargaining chip
In the closing paragraph of my Mac App Store follow-up post, I suggested that eventually most developers will exclusively distribute through the App Store. John Brayton, the developer of CloudPull for Google Docs backup, called this out on Twitter:
"Good post, but disagree that selling outside the MAS won't be worthwhile. IMHO we should be using our own stores as bargaining chip."
In a thread to the MacSB mailing list, John has a related version of this reasoning:
"Selling independently provides protection against Mac App Store policy decisions that could affect my app. If Apple decides tomorrow to kick me out of the Mac App Store, I would take a hit but I would still be able to sell my app."
I couldn't agree more, to both points. There may be some advantages to going App Store-only — less initial setup for checkout and licensing, no confusion about which version to buy, or where to upgrade — but indie Mac developers should be doing everything they can to control their own destiny. Having your own store is just good business sense.
June 26, 2011 09:52 AM [link]
Focusing on the wrong things
Great post from Garrett Dimon, on his biggest mistake building the bug-tracker Sifter:
"I got bogged down watching our bottom-line even though we've always been comfortably profitable. I worried about preventing fraud even though the only instance we ever encountered only cost us $200. I constantly worried that Sifter could go down at any moment even though we've had 99.96% uptime since launch. [...] All of these little things were distracting me from the work that really mattered."
For a small company, focusing on the wrong things will make or break a product. I'm guilty of the same thing. I sat on Tweetmarks for 6 months without launching it because I was worried about how to pay for hosting and how to get developers involved.
Sometimes there's no obvious solution until you ship. Eventually it becomes easier to know when to be patient — to solve a problem right the first time — and when it's needless worrying over something that may or may not even happen. And as 37signals says: decisions are temporary anyway.
June 22, 2011 02:43 PM [link]
Mac App Store follow-up
I've been sitting on this post for a while. First the good news: Clipstart is in the Mac App Store. Overall I was very happy with the response and glad to have a new way for customers to find the app.
I've received a bunch of good feedback on my blog post about Apple's 30% cut. A few people are really upset with Apple, and there are posts in the dev forums about Mac apps that still weren't approved for one reason or another weeks after the store launched. Other developers keep quiet, either for fear of rocking the boat or because they are happy with their sales and don't see a significant problem.
And then there's most of us who know Apple can do even better. We're frustrated when an app (not just our own) is rejected or stuck in review indefinitely, but we just accept that things are a little dysfunctional and cross our fingers that maybe Apple will magically become more transparent.
But it's not going to happen by itself. It's not going to happen because the culture of Apple under Steve Jobs is secrecy. Apple is about great products, sure, but they're so obsessed with the big reveal that it weakens their communication with developers.
From a MacSB mailing list post about WWDC by Dave Howell, written back in February:
"Second, Apple employees are no longer allowed to talk about anything. In the past, half of the value of WWDC was talking directly to the folks who wrote the OS frameworks you have questions about. But now the answer to any question is always either 'file a bug' or 'send an email to devprograms@apple.com.' They're all under a gag order."
The baffling part is that many of the problems in the App Store process are easily solvable. The iTunes Connect team could, for example, make it a priority to answer all email. I don't know what the organizational structure is over there, and I'm sympathetic to what must be a flood of app submissions, but it doesn't feel like App Store support gets the same quality treatment that Developer Technical Support does.
Contrast with Gus Mueller's point on Twitter:
"I'm with you on the 30% + silence issue. With PayPal, they'll call me back when I email them with problems or questions."
Michael Tsai echoes this on his blog:
"The main value of Apple's 30% cut is access to a larger market, but it still doesn't look good that companies such as PayPal, eSellerate, and E-junkie charge much less and provide great service. I can e-mail or call those companies and get answers right away."
Good support takes extra resources and it costs money. Luckily Apple has both, and that's why drawing attention to Apple's 30% cut was key to my original argument. Developers are playing by Apple's rules and helping to fund the App Store.
Despite all this, I'm upbeat. In 2011 I want to look for ways that I can help Apple succeed, such as filing bugs. For years I swore off bothering, because it took so long to turn around a fix, if ever, and I had long since worked around a bug and moved on. iOS changes that delay because it improves so significantly every single year.
I'm all for praising Apple when it's deserved, but history shows that Apple improves the App Store when people complain. My posts are negative when it's warranted and worth paying attention to.
The App Store is getting better. (I love that the Resolution Center is there even if I hope to never need to use it.) The writing is on the wall that a year from now most apps will be distributed through the Mac App Store, and the savings and independence of direct download sales won't be worth the maintenance of two separate forms of distribution for many developers. But if Apple holds all the cards in this relationship, then we must hold Apple to a very high standard.
June 21, 2011 11:22 AM [link]
Carousel
Don Draper, from the season 1 finale of Mad Men (YouTube, skip if you plan to watch the whole series):
"This device isn't a spaceship; it's a time machine. It goes backwards, forwards. It takes us to a place where we ache to go again. It's not called the wheel; it's called the carousel. It lets us travel the way a child travels — around and around, and back home again, to a place where we know we were loved."
When I first saw this a couple years ago I thought of Steve Jobs, the master pitchman in our industry. The delivery is different, more personal here, but it was stunning as part of the full episode. Who doesn't want to build products that resonate so well, that go from nice utilities or productivity apps to something our customers fall in love with?
First you build a product that changes things, that is truly useful. Something ambitious. Then find a way to sell it that connects, and underscore why it matters.
I don't really know how to do this yet. But I do know that part of it is telling stories. Why did I create Tweet Library? To tell stories, to remember events that matter before they're lost in the fleeting stream of old tweets. It's the kind of nostalgia at the heart of the Mad Men clip.
I like this post from Kyle Neath of GitHub, that it's about ideas, not products (via Duncan Davidson):
"People want to be part of ideas. Being part of a company who builds a successful product is cool... but being part of an idea is a lot more attractive. If you can build a business where both your employees and your customers think they're part of an idea, you've created something special."
If you can extract the core idea from a great product, everything that comes next can be matched to the idea, so the product has a clear path for new features. Building a story around it — something that sticks, and having the resources to tell that story properly — takes a lot of work. I'm inspired when I see others do it well, and it's an art I hope to make time for.
June 15, 2011 03:37 PM [link]
WWDC 2011 keynote: diversify
This year's WWDC keynote was one of the most significant of the last few years. Twitter integration and iCloud were the highlights for me, although at the end of the week I'm still not sure when or how I'll be able to use either. But I love that it was a software-only event — that's how WWDC should be — and I love that there were major new features on both of Apple's platforms.
A few of the announcements seemed to have significant overlap (if not direct competition) with third-party developers, in particular Instapaper, Camera+, and the dozens of to-do list apps in the store. You can see some of that live reaction in a collection of tweets I put together at the conference.
My first thought for Marco Arment was that he should come out with a new product. Not because I'm worried about Instapaper, but just because I'd love to see what he'd build next. Marco is still upbeat on Instapaper's chances for continued success:
"If Reading List gets widely adopted and millions of people start saving pages for later reading, a portion of those people will be interested in upgrading to a dedicated, deluxe app and service to serve their needs better. And they'll quickly find Instapaper in the App Store."
Yet here's Dave Winer, reflecting on when Apple competed with his product:
"I think the answer is to find meaning in your work independent of what happens with the fickleness of the platform vendor and its developers. I went on to take the same software that Apple crushed and turned it into blogging, RSS, podcasting, web APIs, all kinds of cool stuff. And yes it did eventually make me a bunch of money. But not the way everyone thought it would."
As Dave used to say, zig where they zag. Find the unique value in the apps you build and spin those out as separate products or use as inspiration for new features. Daniel and I have talked about this on Core Intuition: pull your app's strength into a competition advantage by reusing code and adding more depth than anyone starting from scratch.
By playing to your strengths, you can do more, faster. Every indie Mac and iOS developer should be thinking about a suite of products.
Justin Williams hits this in the context of WWDC:
"Some people grow frustrated by Apple continually making inroads in existing developer's territory, but it comes with being a part of the platform. The key is to ensure your product lineup is diverse enough that you can survive taking the blow Apple may offer at the next keynote."
You should make the choice to diversify before you're forced to make it, because WWDC is already a full year of choices rolled up into one week. Dropbox/Simplenote and iCloud, OAuth and Twitter.framework, iOS 4 and 5, Retain/Release and ARC. Like the Persians deliberating while both drunk and sober (via Buzz Andersen), if you make any real decisions during WWDC's info intoxication, make them again a week later.
Marco has a clear advantage over his new competition, though, regardless of whether he creates new products or sticks with Instapaper. "Send to Instapaper" is built into every great Twitter app and newsreader. It took years to build such widespread integration, and it won't be easy (even for Apple) to be on equal footing with such a well-loved and established brand.
June 14, 2011 01:02 PM [link]
|
