import nest_asyncio
nest_asyncio.apply()

I want to be able to programmatically scrape tweets from my account (and maybe others), push the URLs into a markdown cell, and let fastpages do the work of embedding the tweets in a blog page. I can use twint to get the tweets without using Twitter API. If replies are involved I might need more complicated data structures, but simple tweet-unrolling should be linear and straightforward.

First, I'll fetch the info about a top-level tweet. Then get all tweets from that user, to that user, filtered for conversation_id. TThen print them in chronological order to a markdown display! (Can it be so easy??)

The top level tweet is located at

https://twitter.com/deepfates/status/1412086693713358850?s=20

and should be displayed like

op = 'https://twitter.com/deepfates/status/1412086693713358850'
op_id = op.split('/')[-1]
import requests

r = requests.get(f'https://argh.tweeter.workers.dev/tw/metadata?id={op}')
twt = r.json()
twt
{'conversation_count': 16,
 'created_at': '2021-07-05T16:31:17.000Z',
 'display_text_range': [0, 71],
 'entities': {'hashtags': [], 'symbols': [], 'urls': [], 'user_mentions': []},
 'favorite_count': 42,
 'id_str': '1412086693713358850',
 'lang': 'en',
 'news_action_type': 'conversation',
 'text': "I'm going to rebuild my personal website I think. what should look like",
 'user': {'id_str': '3315205122',
  'name': 'icd ⭐⭐⭐⭐✨',
  'profile_image_url_https': 'https://pbs.twimg.com/profile_images/1406647367546327046/jp7tf-eW_normal.jpg',
  'screen_name': 'deepfates',
  'verified': False}}
import twint
c = twint.Config()
op_uname = twt['user']['screen_name']
op_date = twt['created_at'][:10]
op_id = twt['id_str']
tweets = []

op_uname, op_date, op_id
('deepfates', '2021-07-05', '1412086693713358850')
c.To = op_uname
c.By = op_uname
c.Since = op_date

c.Store_object = True
c.Store_object_tweets_list = tweets

twint.run.Search(c)

self_replies = [tweet for tweet in tweets if tweet.conversation_id == op_id and tweet.reply_to == []]

self_replies.sort(key=lambda x: x.datetime)

urls = [t.link for t in self_replies]
urls
from IPython.display import Markdown as md
templated_links = '>twitter: ' + '\n\n>twitter: '.join(urls)

If I make a raw cell and copy the % include magic from the converted post, I can get it to show up:

And I can make one programmatically, but I can't figure out how to insert that as the next cell.

include = '''{% include twitter.html content='<a href="https://twitter.com/deepfates/status/1412097678759038982">https://twitter.com/deepfates/status/1412097678759038982</a>' %}'''
import nbformat as nbf
raw_cell = nbf.v4.new_raw_cell(include)
from IPython import display
display.display(raw_cell)
{'cell_type': 'raw',
 'source': '{% include twitter.html content=\'<a href="https://twitter.com/deepfates/status/1412097678759038982">https://twitter.com/deepfates/status/1412097678759038982</a>\' %}',
 'metadata': {}}
from IPython.core.getipython import get_ipython
shell = get_ipython()
shell.set_next_input(templated_links)

Well, if i want I can send them into a new cell and manually make it into a markdown cell and run it. That's reasonable. Alternatively, I could insert them into a new notebook entirely, as raw includes.

The other problem is that this Twitter embed includs the previous tweet, so the idea of posting abuch ofthem as a thread is kind of awkward. Might work better for a "popular tweets" post?

Yes... In fact, it could even be a notebook I run every so often taht prints to a .md file in the pages folder. All it has to have is YAML frontmatter and a list of includes...