FacebooLiveSearch-URL2

Facebook Live Map: Video Search

Facebook’s Live Map contains everything you need to find a live stream with one exception: It’s a visual display without a search function.

In my last post, I identified the data that populates the map’s live stream viewer locations. In this post, I’ll look at who is streaming the videos (called “publishers” by Facebook) and introduce a bookmarklet and a search tool I created to display their information: the publisher names, their business categories and locations, plus the descriptions, viewer counts and previews of the videos being streamed. I’ll also share some thoughts about preserving and downloading the streams and the viewer comments that you’ll encounter.

 

The Problem with Facebook Search

When I need to find a person, place or video on Facebook, my natural inclination is to search for it using  Facebook Graph Search. It’s the logical direction to take, except when the thing you’re looking for is a live stream video. And, that’s because live stream search doesn’t exists.

Part of the problem is that Facebook does not have a query language that distinguishes between “live streams” and “videos.” They’re actually two different things.

Live streams are real-time broadcasts. They are delivered in chunks that Facebook assembles in the video player. When the broadcast ends, Facebook assembles and converts the chunks into a physical mp4 file. That’s the file that you can download or play back after the stream ends.

Facebook Graph knows how to find videos and connect them to profiles, locations and dates. The problem is we don’t have the same type of query language for video streams. We can try to make Graph work, but we’ll get a lot of search results that don’t match our intention. Let’s illustrate this by targeting videos in different ways.

We learn about videos, live streams and photos through posts. If we focus our search on posts that are “live” or dated “today,” we get all of these things combined together.

Live posts with the Keyword “New”
FB.com/search/str/news/stories-keyword/stories-live/intersect/

Posts dated Today with the Keywords “Live” “News” 
FB.com/search/str/live+news/stories-keyword/stories-news/today/date/stories/intersect/

Hashtags are sometimes found within posts and video descriptions, but they are even less reliable for live search because they aren’t filtered by date

Hashtags for “News”
FB.com/hashtag/news/

Facebook search works best when you’re looking for Objects that were created to make connections like Profiles, Places, Photos, Videos and Dates. Facebook doesn’t have an Object named Live Stream, so we actually have better luck using search to find them in their past tense: as completed Videos.

Videos by a user with the keywords “was live”
FB.com/search/str/was+live/videos-keyword/4/videos-by/intersect/

Videos in a location with the keywords “live”
FB.com/search/str/live/videos-keyword/110970792260960/videos-in/intersect/

 

The Problem with Facebook Live Map

Facebook’s Live Map is clearly built from data. We just have to figure out how we can search it.

When we look at the map, we see dots representing the profiles of video publishers. When we hover over a publisher, we see details: the name of the video publisher, their business category if it’s a page, the current number of viewers, a preview of the video and its current duration.

Facebook Live Map

 

Live Map is visually interesting, but it’s not practical when your goal is to find publishers or content. There are too many dots to inspect and it’s difficult to select dots located within a highly populated area or the site of a breaking event.

That’s why we want to use this same data to build a different display like a list. With it, we can search and find streams by keywords, publisher names, hashtags, and locations. To do this, we just need to find the URLs that populate the map with this information.

 

How to View Facebook Live Video Details

Facebook continually updates and builds the video publisher display on Live Map through three URLs. I’ll refer to them by their labels: Level 0, Level 1, and Level 2.

Facebook Live Map Video Publisher URL

To view a Level 0-2 URL, you will need to begin on the Facebook domain with an active login. You’ll add your Facebook ID  to one of the URLs below, press enter and that’s it. You’re viewing the data for a batch of publisher streams.

Level 0
FB.com//ajax/livemap/map/data/?level=0&video_count=500&dpr=1&__user=USERID&__a=1&

Level 1
FB.com/ajax/livemap/map/data/?level=1&video_count=500&dpr=1&__user=USERID&__a=1&

Level 2
FB.com/ajax/livemap/map/data/?level=2&video_count=500&dpr=1&__user=USERID&__a=1&

 

Note: I’ve increased the video count number in the URL from 400 to 500 so that I can capture a couple extra streams. The largest number of streams I’ve seen displayed in a batch so far is 417.

The publisher data changes each time you refresh your browser. The publisher that was at the top of the page will drop down a notch or two as other streams appear above it. These streams can be newer or older than the publisher they displaced. Their viewer counts will also vary. There’s no real pattern here. Just three different batches of publisher data managed on three URLs.

Try copying and placing the data into a JavaScript beautifier. This will organize your view and you’ll begin to see each publisher and the fields detailing their stream, just like the image shown below. We can use these fields to create our new display.

Facebook Live Video Publisher Data

These fields include:

  • “name” –  Video publisher’s Profile or Page name
  • “startTime” – Video start time in epoch or machine time
  • “previewImage” – Video thumbnail image
  • “viewerCount” – Current viewer count
  • “publisherCategory” – Primary category assigned to the Page
    This is “null” for personal Profiles
  • “videoID” – Unique ID – Also used in links, i.e. FB.com/VIDEOID
  • “message” Post or video description
  • Hashtags
  • “url” – Publisher’s video URL
  • “lat” “long” – Publisher’s location from their “About” page

 

Note: As with viewer locations, the publisher’s Lat/Long coordinates are obtained from the residences or business addresses listed on their About page, or by tagging the location like a visitor check-in. There are instances where this will be misleading. For example, a Peruvian publisher will appear to be streaming from Los Angeles on a Facebook Live Map if their profile, page or check-in is set to that location.

 

Facebook Live Stream bookmarklet

Displaying this data is relatively straightforward. The easiest way to do this is to create a JavaScript bookmarklet that extracts the data from the URL and displays it as an HTML table on top of a Facebook page. To keep it simple, I’ll use separate bookmarks for the three different Levels.

The script to generate this display already exists thanks to a Facebook typeahead bookmarklet created by @jkeesh and @karmiphuc. I’ve simply inserted our Live Map field names in place of the fields from their tool which, by the way, provides a great insight into what you search and how Facebook ranks it in your search suggestions.

Facebook Live Search Bookmarklet

My modification of the typeahead bookmarklet, as well as the Facebook Live bookmarklets, is available here on NetBootCamp.org. You can see a display from the Facebook Live bookmarklet above listing the publishers, their page categories, the stream descriptions and their locations. I’m still working on features that include video image previews, links to download the videos, and maps to the publisher locations. I’d look forward to sharing your improvements, as well.

You’ll also find a comprehensive set of Facebook Live and Video queries and tools on a dedicated page here on NetBootCamp.org. Among other things, this tool will help you find live streams by keywords and location.

 

Preserving and Downloading Facebook Live Videos

Live streams are fleeting. You’ll never know how long it’s going to last or whether the video will be saved at the end of the broadcast. That may encourage you to preserve the video data as you see it.

Live streaming delivers videos in chunks via an m4v format. When the stream concludes, Facebook assembles and converts the chunks into an mp4 format.  You can view the playlist pointing to the stream in progress and the mp4 file created after its conclusion through the URL below, provided the publisher doesn’t delete it.

Live Video Stream and Download Link
FB.com/video/video_data/?video_id=VIDEOID

You can get the VideoID using a NetBootCamp bookmarklet or you can “join” the stream by selecting the hyperlinked duration time. This will redirect you to a URL constructed like the one shown below. Just note that the video publisher will be able to see your profile name when you join.

FB.com/PUBLISHER/videos/VIDEOID

The VideoID is the key to many things. You can use it to find the publisher’s profile and the page that will ultimately host the video by placing the ID in a URL next to FB.com.

FB.com/VideoID
FB.com/10102776370897261

Now that you’ve found a live stream and its publisher, let’s say you want to preserve the video stream as evidence. Live stream preservation is a complicated process. It requires practice and some set up if you’re doing this manually. We’ll cover this in more detail in a future post, but for now consider these tips:

Live Stream Video and Comment Preservation Tips

1) Don’t refresh the page
You can usually hold a video in your cache after it’s been deleted provided that you don’t refresh the page. Open one browser to allow the live stream to buffer and cache. Use a separate browser or window to research and explore the video data and the publisher’s profile. When you’re done, you should be able to return to the first window to rewind and replay the stream. It’s a good safeguard for any stream containing critical or questionable content that the publisher might not want to leave online. You might not be able to recover the full video from your cache when the publisher deletes it.

2) Capture short-run buffered videos via desktop recording

You can retrieve videos from your browser’s cache using tools like Nirsoft’s VideoCacheView. You will only be able to save the parts of the video you actually viewed and, as a stream, there will be many parts to reassemble. I’ve had limited luck recovering Facebook streams this way.

Streams are hard to capture on-the-fly. That’s why I suggest using a desktop recorder like Camtasia. It’s easy to launch. You could do your research during the stream in a second computer monitor, but I would recommend recording your buffered window later. That’s because the combined memory consumption of the recorder and your browser can cause the browser to freeze or even crash. When the browser crashes and closes, your video buffer goes with it.

3) Capture backup copies with VLC player
The best quality video is going to be captured direct from the streaming source. This can be done using the stream’s m3u8 file (located on the video download link) and VLC Player. This is much more complicated than launching a desktop recorder and you won’t know if you failed until the stream ends.

One of the downsides to this method is that you’ll never capture the full video file. You’ll be capturing the video from the point in time you intercepted the stream. There’s no rewinding because those files have already passed by. VLC is a good backup option for streams that may run long. I’ll provide a separate post on this later.

4) Preserve the Likes, Comments and Viewer IDs
You may also want to preserve the information about the viewers while the stream is live. You can archive the page and run the risk of freezing the browser or you can save a backup copy of the viewer data. You can save the comments data by opening the URL below with your Facebook ID.

FB.com/videos/livecomments/?eft_id=VIDEOID&dpr=1&__user=USERID&__a=1

The advantage to this data is that it also includes the comment count, names, genders and fbids of  the commenters along with the text of their comments and their friendship status with the publisher. This data constantly updates, so do not refresh the browser when the broadcast ends or you will lose it.

You can also open another URL to preserve the Likes, Reactions, and Shares during the broadcast. This may also help identify the relationship between the broadcaster and their audience. Some of these viewers may be Friends or Followers of the publisher.

Likes & Reactions 
FB.com/ufi/reaction/profile/browser/?ft_ent_identifier=VIDEOID&av=USERID

Shares 
FB.com/shares/view?id=VIDEOID

5) Anonymity

Publishers see the viewer count increase when you preview their live stream via your news feed or the Live Map. They shouldn’t see your profile name until you “join” the stream. Users join streams by viewing the URL hosting the video. The publisher will also see your  profile picture next to your name but, in my recent tests, the images were not accompanied with a link to the viewer’s profile page.

Publishers can also enjoy some anonymity. They can stream as a Page without revealing their personal profile. They can also stream to a custom list of friends or a group that is secret or closed to the public. Secret groups are not searchable, so it can be worth joining a group before it transitions to the secret category.

 

Summary
Get the most out of Facebook Live. Experiment with the Live Map URLs to explore and monitor streams by keywords, publishers, categories and locations.

Tools: Facebook Bookmarklets | Facebook Live Search Tool