Importing tweets to Micro.blog

Today we launched a new feature: importing an archive of your tweets to Micro.blog. This is available to all paid subscriptions and can be accessed on the web under Posts → “…” → Import.

Very early Micro.blog customers may remember that we used to have a Twitter import feature alongside our traditional blog import from WordPress, Tumblr, and other platforms. We disabled this years ago because I kept hearing from people who realized that it was a mess importing tens of thousands of tweets into their new blog. I’ve taken that feedback to build this new feature into something that I think works much better.

There are a few unique twists with how this works:

  • When you import your tweets, we create a new blog in your Micro.blog account just for the tweets. This is in the format username-tweets.micro.blog. There’s no extra charge for this blog.
  • This new blog stores all your tweets as blog posts. Micro.blog also copies any photos in tweets to this tweets blog.
  • A new plug-in is installed that links your existing blog with the tweets blog. This plug-in provides a streamlined interface for browsing tweets by month or searching across all your tweets.

You can see my own tweets on my blog here: manton.org/tweets

Structuring the tweet storage this way means we can leverage many of the powerful features built in to Micro.blog:

  • There’s a new API that the plug-in uses. You can make your own version of the plug-in from GitHub by copying its HTML and JavaScript. You can even access the API of tweets from apps and blogs outside of Micro.blog.
  • The separate tweets blog is a full Micro.blog-hosted blog so you can use custom Hugo templates to make your own web interface for browsing tweets, or add your own domain name to it.
  • If you have multiple Twitter accounts, you can create additional blogs for those accounts. The trick is to name the blog with a “-tweets” suffix. (We only include one extra tweets blog for free. The rest will be added to your subscription just like any additional blog.)

Because the storage is just blogs, you can use existing apps to manage the tweets and photos. For example, you can use MarsEdit to download and search all your tweets. It’s also easy to zap all your tweets and start over without touching your actual blog.

Some limitations: I’ve decided to only copy photos from the archive, not other types of media. If you have a lot of photos, expect this to take a long time as Micro.blog extracts the photos from your archive and uploads them to our servers. You can keep an eye on what it’s doing on Account → View logs. I expect we may tweak this as we get feedback from more people.

Enjoy! I hope this makes it a little easier to say goodbye to Twitter.

Patrick Rhone

This is more than a bit brilliant.

Brandon Bohling

awesome, thanks! Since it is separate from the main micro.blog I’m assuming there is no issues with those that have been cross-posting from here to Twitter for years, right?

Manton Reece

@bbohling Yep, that's fine. It doesn't attempt to figure out which tweets started as blog posts either.

Manton Reece

@patrickrhone Thanks!

Luke M. Perez

Splendid, Manton. Just Splendid.

Pratik

Fantastic. Trying it now. Does it matter if I don't have the original zip file but rather the unzipped folder? My Twitter account no longer exists so I can't redownload the archive.

Manton Reece

@pratik That's fine as long as the format for the archive matches the new archives from Twitter. If it's years old, Twitter may have changed the format, so I'm not sure.

Pratik

Ok. I just zipped it back up and tried the import. It didn’t throw any errors but don’t see anything yet 🤞🏽

Munish Pruthi

wow this is amazing.

David Johnson

While I have no use for this new feature, I marvel at what you have created. Incredible!

Pratik

Oh well, it didn't work. It did not throw up an error, but it created the other blog as it should, but it shows zero posts.

Manton Reece

@pratik Must be a different format. I'll add some checks for that. Do you want to send me a screenshot or description of what the files/folders inside the .zip look like?

Dr. Adam Procter

happy days just importing :)

Pratik

Sure. This is what I have:

This was downloaded July 2018.

Manton Reece

@pratik Yep, that’d the old format. Hmm. I’ll look into whether we can support both.

Pratik

Thanks. That would be great if you could.

Khaled Abou Alfa

@crossingthethreshold share the sentiment. Clever workaround. Zero interest in any of it. Can’t complain, we got AI transcripts for podcasts and the archive link feature (which is excellent).

I am due a what I’d like to see in Micro.blog type post…(spoiler: more Twitter features won’t feature in this).

Bob Wertz

Micro.Blog is chugging away on my import right now. Going well so far.

Anton Zuiker

I have Twitter archives circa 2021, 2019, 2017, 2014, and maybe earlier and the format definitely changed a few times over that period. The bulk of my tweets (2007-2014) are in one of these older formats, and that's what I'd love to import. Thanks anyway & always.

Nick Broughall

This is awesome! I've been holding out, hoping I'd be able to bring across my archive at some point. Thanks for making it so easy!

Manton Reece

@supremus No, I decided to make all of this work together that the tweets will need to be public. Even if you don't share a link to them, they are technically accessible via the API, so if there is private stuff in the tweets I recommend deleting the -tweets blog completely. (Private posts is something we will revisit later, but not this year.)

Ricardo Mendes

waiting for my archive to be downloadable, really curious about this so I took a paid plan!

By the way micro.blog evolved so much since last I checked, it's amazing work!

Manton Reece

@rmdes Thank you! It has come a long way. Let us know if you run into any tweet problems.

Ricardo Mendes

how does it work if in the future I import another archive, same user, will it just do incremental and add up the the archive provided it's the same user?

Ricardo Mendes

how does it work if in the future I import another archive, same user, will it just do incremental and add up the the archive provided it's the same user?

Manton Reece

@rmdes When importing again it clears the previous tweets first. There's probably some room for improvement here to make it more efficient.

Ricardo Mendes

Interesting, thanks! yeah maybe taking some ideas from here github.com/tweetback... would allow the setup to be connected to a twitter app and then periodically just index new ones, like tweetback is doing :)

Manton Reece

@rmdes What Tweetback is doing there is going to stop working this month with the Twitter API changes. There will no longer be free read access to Twitter. 🙁

Ricardo Mendes

oh shoots.. I understand, that's really sad... And not surprising #theMollusk

Ricardo Mendes

I'm importing my zip file since yesterday.. It's like 99% loaded but it seems to be stalled, anything that could be done server side to execute the import? Or should I try the upload again?

Manton Reece

@rmdes I've seen a case of the import just stalling and never finishing. If it looks like that happened (check the Account → View logs) I recommend just uploading again. It will clear the tweets and start again, but it won't have to copy the photos again, so should be much faster.

Ricardo Mendes

ongoing it's at the same spot 99%

Manton Reece

@rmdes I think I misunderstood, is this the progress bar during upload, or are you guessing the 99% because of how many tweets have loaded? If you don't mind, follow up with an email to me at help@micro.blog and I'll investigate in more detail.

Ricardo Mendes

not seeing anything related to this in the log but there is some token error, maybe from a mobile app (I'm trying different mobile microblog clients)

Ricardo Mendes

progress bar, I can see that it's all loaded up less a very tiny small white space, OK I'll drop an email!

Ricardo Mendes

Good news, even with an untouched 1.7GB Twitter archive!!!

Apparently it's processing, see your email for log :))

Ricardo Mendes

Good news, upload worked even with an untouched 1.7GB Twitter archive!!!

Apparently it's processing, see your email for log :))

edit:I'v let it do it's thing the entire night but nothing appears in the tweet blog, so now I'm uploading again let's see if it manages to crunch the archive

Ricardo Mendes

just an update on this, I cleaned my Twitter archive from duplicates and deleted tweets media, went from 1.7GB to 1.2GB uploaded, now it's importing and I can see it's working on the log just fine !! waiting to complete to check the results !! big thanks for this feature !

Manton Reece

@rmdes Excellent, glad it’s working!

Martin Feld

Hi, ! Hope you've had a great day! I'm wondering if you could help with this (or if anyone else who's followed this thread has seen it): after importing all of my tweets into a new tweet-specific Micro.blog, I can see that my site's archive now shows all posts whenever you click on a specific category, which it didn't before. I successfully imported all tweets and thought that this would resolve itself, but it hasn't. Did I do something wrong or is there some way that I can fix it, so that each category only shows its specific posts? Thanks! 😀

Manton Reece

@martinfeld Has anything else changed with your blog settings or theme? The tweets really shouldn't have any affect on categories on your blog. I feel like it must be coincidence. Might be worth trying to change to a different theme plug-in temporarily to see if that fixes it.

Martin Feld

Thanks for the quick response, although I didn't expect it! I just saw that you had a tweet plugin update, and so I did that and now only one archive category seems to be affected—looks good! It's actually a good excuse to go and fix up the silliness that is my categories, so thanks again!

Manton Reece @manton
Lightbox Image