How to use the Virtual Machine for Researchers

Posted on July 4, 2013 by brewster

Some researchers that are working with the Internet Archive, such as those at University of Massachusetts, have wanted closer access to some of our collections. We are learning how to support this type of “on-campus” use of the collections. This post is to document how to use these machines.

Who can have access?

This is for joint projects with the archive, usually some academic program often funded by NSF.  So this is not a general offering, but more of a special case thing. Most use the collections by downloading materials to their home machines. We have tools to help with this, and use “GNU Parallel” to make it go fast.

How to get an account?

Is there an agreement? Yes, there usually is. This is usually administered by Alexis Rossi.  All in all, these are shared machines, so please be respectful of others data and use of the machines.

How do I get access to the VM? To get an account you will need to forward a public SSH key to Jake Johnson. Please follow the steps below for more details.

Generate your SSH keys.

  1. If you don’t already have an ~/.ssh directory, you will need to create one to store your SSH configuration files and keys: 
    $ mkdir -p ~/.ssh
  2. Move into the ~/.ssh directory: 
    $ cd ~/.ssh
  3. Create your keys (replacing {username} with the username you would like to use to login to the VM): 
    $ bash -c 'ssh-keygen -t rsa -b 2048 -C "{username}@researcher0.fnf.archive.org"'
  4. You will be prompted to enter a filename which your private SSH key will be saved to. Use something like id_rsa.{username}@researcher0.fnf.archive.org, again replacing {username} with your username that you will be using to login to the VM): 
    Enter file in which to save the key (~/.ssh/id_rsa): id_rsa.{username}@researcher0.fnf.archive.org
  5. You will be prompted again to enter a passphrase. Enter a passphrase, and continue. 
    Enter passphrase (empty for no passphrase): [enter your passphrase]
Enter same passphrase again: [enter your passphrase again]

You should now have two new files in your ~/.ssh directory, a private key and a public key. For example:

~/.ssh/id_rsa.{username}@researcher0.fnf.archive.org
~/.ssh/id_rsa.{username}@researcher0.fnf.archive.org.pub

Your public key is the key suffixed with “.pub“.

Adding your public key to the VM

Forward your public key  to Jake Johnson. He will create a user for you, and add your public key to the VM. Once you receive notification that your user has been created and your key successfully added to the VM, proceed to the next step.

Logging into the VM via SSH

You can now use your private key to login into the VM with the following command:

$ ssh -i ~/.ssh/id_rsa.{username}@researcher0.fnf.archive.org {username}@researcher0.fnf.archive.org

How do I bulk download data from archive.org onto the VM?

We recommend using wget to download data from archive.org. Please see our blog post, Downloading in bulk using wget, for more details.

If you have privileges to an access-restricted collection, you can use your archive.org cookies to download data from this collection by adding the following --header flag to your wget command:

--header "Cookie: logged-in-user={email%40example.com}; logged-in-sig={private};"

(Note: replace {email%40example.com} with the email address associated with your archive.org account (encoding @ as %40), and {private} with the value of your logged-in-sig cookie.)

You can retrieve your logged-in-sig cookie using the following steps:

  1. In Firefox , go to archive.org and log in with your account
  2. Go to Firefox > Preferences
  3. Click on the Privacy tab
  4. Select “Use custom settings for History” in drop down menu in the history section
  5. Click the “Show cookies” button
  6. Find archive.org in the list of cookies and expand to show options
  7. Select the logged-in-sig cookie. The long string in the “Content:” field is the value of your logged-in-sig cookie. This is the value that you will need for your wget command (specifically, replacing {private} in the --header flag mentioned above).

How do I bulk download metadata from archive.org onto the VM?

You can download all of an items metadata via our Metadata API.

How do I generate a list of identifiers for downloading data and metadata from collections in bulk?

You can use our advanced search engine. Please refer to the Create a file with the list of identifiers section in our Downloading in bulk using wget blog post.

How can I monitor usage of the VM?

You can monitor usage of the VM via MRTG (Multi Router Traffic Grapher) here: http://researcher0.fnf.archive.org:8088/mrtg/

Posted in Technical | Tagged | 2 Comments

The Internet Archive Metadata API

Posted on July 4, 2013 by internetarchive

The Metadata API is intended for fast, flexible, and reliable reading and writing of Internet Archive items.

Metadata Read API

The Metadata Read API is the fastest and most flexible way to retrieve metadata for items on archive.org. We’ve seen upwards of 500 reads per second for some collections!

Overview

Returns all of an item’s metadata in JSON.

Resource URL

http://archive.org/metadata/:identifier

Parameters

identifier: The globally unique ID of a given item on archive.org.

Usage

For example, frenchenglishmed00gorduoft is the identifier for http://archive.org/details/frenchenglishmed00gorduoft. You can retrieve all of this item’s metadata from the Metadata API using the following curl command:

$ curl http://archive.org/metadata/frenchenglishmed00gorduoft

The Metadata API also supports HTTPS:

$ curl https://archive.org/metadata/frenchenglishmed00gorduoft

Sub-item Access

The Metadata API returns all of an item’s metadata by default. You can access specific metadata elements like so:

http://archive.org/metadata/:identifier/metadata

http://archive.org/metadata/:identifier/server


http://archive.org/metadata/:identifier/files_count


http://archive.org/metadata/:identifier/files?start=1&count=2


http://archive.org/metadata/:identifier/metadata/collection


http://archive.org/metadata/:identifier/metadata/collection/0


http://archive.org/metadata/:identifier/metadata/title


http://archive.org/metadata/:identifier/files/0/name

Metadata Write API

The metadata write API is intended to make changes to metadata timely, safe and flexible.
It utilizes version 02 of the JSON Patch standard.

Overview

timely

  • Callers receive results (success or failure) immediately.
  • Changes are quickly reflected through the metadata read API.

safe

  • All writes pass through the catalog, so all changes are recorded.
  • All writes are checked before they’re submitted to the catalog.
  • If there’s a problem, no catalog task is created. Goal: no redrows!
  • All checks are repeated when the catalog task is executed.

flexible

  • Supports arbitrary changes to multiple metadata targets through a unified API.
  • Changes are easy — no string concatenation or libraries needed.

Resource URL

http://archive.org/metadata/:identifier

Parameters

identifier: The globally unique ID of a given item on archive.org.

Targets

The Metadata Write API supports three kinds of target:

metadata: Changes item_meta.xml (e.g. http://archive.org/metadata/:identifier/metadata).
files/:filename: Changes the file entry in the item’s files.xml (e.g. http://archive.org/metadata/:identifier/files).
other: Changes other.json (e.g. http://archive.org/metadata/:identifier/other).

For XML targets (e.g. ‘metadata‘ and ‘files‘) patches should be composed against their JSON representation, as found in metadata read API results.

Usage

As an HTTP post/get

http://archive.org/metadata/:identifier

With the following url-encoded arguments:

-target: The metadata target you would like to modify.
-patch: The patch you are submitting to the Metadata API.
access: Your IA-S3 access key.
secret: Your IA-S3 secret key.

Authentication

NOTE: These calls must be made with appropriate authentication – at the moment, this means passing your Archive.org IA-S3 credentials. Please visit http://archive.org/account/s3.php to obtain your IA-S3 access key and secret key.

Patches

Patches are JSON strings. They should comply to the draft Json-Patch standard:

http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-02

Examples

Writing to an item’s meta.xml

Add ‘scan_sponsor’ with value ‘Starfleet’ to target ‘metadata’ to the item metadata_test_item:

#!/bin/bash
ACCESS=<redacted>
SECRET=<redacted>
IDENTIFIER=metadata_test_item
TARGET=metadata
PATCH='{"add":"/scan_sponsor", "value":"Starfleet"}'

curl --data-urlencode -target=$TARGET \
     --data-urlencode -patch="$PATCH" \
     --data-urlencode access=$ACCESS \
     --data-urlencode secret=$SECRET \
     http://archive.org/metadata/$IDENTIFIER

returns a JSON object, like the following:

{"success":true,"task_id":114350522,"log":"http://www.us.archive.org/log_show.php?task_id=114350522″}

or perhaps

{"error":"Some problem applying the patch"}

writing to files.xml entry

#!/bin/bash
ACCESS=<redacted> 
SECRET=<redacted>
IDENTIFIER=metadata_test_item
TARGET='files/glogo.png'
PATCH='{"add":"/camera", "value":"Canon A150″}'

curl --data-urlencode -target=$TARGET \
     --data-urlencode -patch="$PATCH" \
     --data-urlencode access=$ACCESS \
     --data-urlencode secret=$SECRET \
     http://archive.org/metadata/$IDENTIFIER

Writing to metadata_test_item/foo_client.json

NOTE: Keys and values are binary-safe and unrestricted

#!/bin/bash
ACCESS=<redacted> 
SECRET=<redacted>
IDENTIFIER=metadata_test_item
TARGET='foo_client'
PATCH='{"add":"/of concern to foo", "value":{"foo-ness":["buckle", "shoe"]}}'

curl --data-urlencode -target=$TARGET \
     --data-urlencode -patch="$PATCH" \
     --data-urlencode access=$ACCESS \
     --data-urlencode secret=$SECRET \     
     http://archive.org/metadata/$IDENTIFIER

After the above call, a metadata read of metadata_test_item will have a toplevel member ‘foo_client’ with value:

{"foo-ness":["buckle", "shoe"]}
Posted in Technical | Tagged , , | 1 Comment

Cost to Store All US Phonecalls Made in a Year in Cloud Storage so it could be Datamined

Posted on June 15, 2013 by brewster

(Cited in Forbes)

Because of recent news reports, I wanted to cross check the cost feasibility of the NSA’s recording all of the US phonecalls and processing them.

These estimates show only $27M in capital cost, and $2M in electricity and take less than 5,000 square feet of space to store and process all US phonecalls made in a year.   The NSA seems to be spending $1.7 billion on a 100k square foot datacenter that could easily handle this and much much more.    Therefore, money and technology would not hold back such a project– it would be held back if someone did not have the opportunity or will.

Another study concluded about 4x my data estimates others have suggested the data could be compressed 10:1, and the power bill would be lower in Utah.   A Google Doc version of the spreadsheet and a cut and past version below.

This was just boingboing’ed.

number of call-minutes per person per month 300 minutes (estimate from my family’s usage)
sides in a phonecall (caller+receiver) 2 since most calls are domestic, only need to record a call once for each reciever/caller pair
number of people in the US 315,000,000 https://www.census.gov/
number of bytes/sec in a phonecall 8,000 this is the uncompressed number, could be compressed to 1/2 to 1/4 easily
cost of a Petabyte (PB) of “cloud” storage $100,000 this is basically what the Internet Archive pays. Petabyte = 1,000 terabytes
Square feet of datacenter space per petabyte 16 2 feet wide by about 8 feet including corridor between racks
Power to run a PB 5 kilowatts
Cost per KWhr $0.15 California costs (higher than much of the country, could be 1/2 in other places)
number of bytes/min in a phonecall 480,000 calculated from above
number of bytes/month for a person 144,000,000 calculated from above
number of bytes/month for the US 22,680,000,000,000,000 calculated (divided by 2 because there is a caller and reciever, don’t need to double count)
number of PB/month for the US 23 calculated
number of PB/year for the US 272 calculated
Cost to store all phonecalls made in a year in the “cloud” $27,216,000
Square feet to store all phonecalls 4,355
Cost of datacenter power for all phonecalls for a year $1,788,091
Posted in Announcements, News | 40 Comments

Knight Foundation Strengthens Support for Television News Research Service

Posted on May 21, 2013 by Roger Macdonald

Thanks to a recent $1 million grant from the John S. and James L. Knight Foundation, we will be expanding our TV News Search & Borrow service that enables everyone to search, quote and borrow U.S. television news programs.

Launched last September, the service repurposes closed captiFront Page frackingoning to facilitate deep search and present relevant short-streamed with clips from more than 400,000 news broadcasts dating back to June 2009. We are striving to help inform and engage communities by strengthening the work of journalists, scholars, teachers, librarians, civic organizations and others dedicated to serving public interests.

We are beginning to see important public benefits arising from this new capability to apply digital search and aResults fracking 2nalysis to news from our most pervasive and persuasive medium—television. Journalists are better able to investigate significant persons and events. Documentarians are more effectively finding key news footage to license and use. Educators can now focus the critical attention of their students on extensive real-world examples of how news stories are told and audiences engaged.

We recently worked witTrayvonh researchers at Harvard’s Berkman Center and MIT’s Center for Civic Media to facilitate direct machine queries of our television news library that returned structured data results to inform their media landscape analysis of the Trayvon Martin story and reveal key pivot points in its evolution.

 

Journalists and documentarians at the newly-launched Retro Report are using TV News Search & Borrow to help them take a fresh look at important stories of the past, share new perspectives and add insightful commentary to what are sometimes all too shortsighted first drafts of history.

We are also working with a number of scholars, journalists and civic organizations to see how our research library might help improve political accountability and transparency by indexing television political advertising and pairing them with information on ad sponsors from FCC-mandated “public inspection files” at each station.

Daisy_Ad_1964   “Daisy“

Such a special collection could also be used to study interactions between campaign messaging and local news coverage. The 2013 elections in Virginia, a state with no political campaign contribution limits, may be a useful test-bed for experiments like these.

We are following up on suggestions from media professionals that a comprehensive research library of local television news might also better inform stations and their audiences about how programs are helping to meet the critical information needs of local communities.

VanderbiltOur TV News Search and & Borrow service preserves and makes responsibly accessible an enduring library of television news, serving important public benefit research interests of today and those of generations to come.  In doing so, it stands on the shoulders of the pioneering work of Vanderbilt University’s Television News Archive and, more recently, UCLA’s NewsScape library.

We are humbled by the challenges of exploring the new territory of scaling intelligent access to our growing digital public library of television news and welcome feedback on how we can better serve the public interest.

Posted in Announcements, News | 3 Comments

National Security Agency ❤ ❤ ❤ Internet Archive?

Posted on May 18, 2013 by brewster

nsa_logo_2An unclassified document from the National Security Agency from 2007 has some nice words to say about the Internet Archive, Brewster Kahle, and the Wayback Machine.

“The Wayback Machine is, very simply, one of the greatest deep web tools ever created.” -National Security Agency (2007)

https://www.nsa.gov/public_info/_files/Untangling_the_Web.pdf

A searchable version, and a searchable PDF version.

Main section on us:

The Internet Archive & the Wayback Machine

You have to give Brewster Kahle credit for thinking big. The founder of the Internet Archive has a clear, if not easy, mission: to make all human knowledge universally accessible. And, who knows, he might just succeed. What has made Kahle’s dream seem possible is extremely inexpensive storage technology. As of now, the Internet Archive houses “approximately 1 petabyte of data and is currently growing at a rate of 20 terabytes per month. This eclipses the amount of text contained in the world’s largest libraries, including the Library of Congress. If you tried to place the entire contents of the archive onto floppy disks (we don’t recommend this!) and laid them end to end, it would stretch from New York, past Los Angeles, and halfway to Hawaii.” 102 In December 2006 the Archive announced it had indexed over 85 billion “web objects” and that its database contained over 1.5 petabytes of information. 103

But that’s not all that Kahle and company have archived. The Archive also now contains about 2 million audio works; over 10,000 music concerts; thousands of “moving images,” including 300 feature films; its own and links to others’ digitized texts, including printable and downloadable books; and 3 million hours of television shows (enough to satisfy even the most sedulous couch potato!). Kahle’s long term dream includes scanning and digitizing the entire Library of Congress collection of about 28 million books (something that is technically within reach), but there are UNCLASSIFIED  some nasty impediments such as copyrights and, of course, money. None of this deters Kahle, whose commitment to the preservation of the digital artifacts of our time drives the Internet Archive. As Kahle puts it, “If you don’t have access to the past, you live in a very Orwellian world.”

 

Posted in Announcements, News | 22 Comments

Brewster Kahle to be Honored with 2013 Amer Lib Assoc LITA/Library Hi Tech Award

Posted on May 10, 2013 by brewster

Brewster Kahle is honored to receive the 2013 LITA/Library Hi Tech Award for Outstanding Communication in Library and Information Technology this year.   It will be awarded at the American Library Association meeting in Chicago in June.

http://www.ala.org/news/pr?id=13005

 

Posted in Announcements, News | 2 Comments

Free and Fast ‘Roof2Roof’ Internet Available in Richmond, CA

Posted on May 8, 2013 by brewster
Antenna on 2512 Florida Avenue, Richmond to offer free Internet for those with antennas on their roofs

Antenna on 2512 Florida Avenue, Richmond to offer free Internet for those with antennas on their roofs

As a free service to Richmond residents, the Internet Archive has installed a 70 foot tower on its physical archive building in Richmond California to offer free and fast Internet to those with roofs that can see the tower.  Those wanting to use this community wireless service would need to buy and install a directional antenna on their roof to connect, but from then on their Internet access is free.   In this way we call it a ‘free and fast roof2roof network’ since it will generally not reach people’s laptops inside houses.   The signal will work at over 1 mile to a suitable antenna with line-of-site to our tower.    Wifi receivers with directional antennas can cost as little as one hundred to two hundred dollars from vendors like ubiquiti.

Gayle McLaughlin, mayor of Richmond, when we told her about this, said: “We are dedicated to closing the digital divide in Richmond. Providing free access to the internet is a great benefit for our residents helping us create a better and more equitable city!”

End-user window mountable antenna for connecting to Internet Archive's tower

End-user window mountable antenna for connecting to Internet Archive’s tower

We have achieved 80 megabits per second in both directions with this technology, so this should support many people’s normal Internet use.    Typical commercial Internet access runs at 1/10 this speed, so the fastest residential Internet in Richmond will likely be this system.    Currently average of 4 users are connect to our tower but we hope this will grow.

We hope that intrepid individuals will connect to this system in a way we have called “tier 3″.   While we do not have the budget to provide tech support, we hope that entrepreneurs, enthusiasts, or non-profit organizations will help others get online.

Another step would be to expand the number of houses and buildings that could connect to this system by putting repeater antennas on high locations to expand the number of rooftops with line-of-site to this backbone.    If you are an owner of a tall building or structure and are interested in participating, please let us know by writing to info@archive.org.   We would be interested in paying for the equipment and do the installation for a couple of well placed locations.

Location: Height 70′ above ground level, 2512 Florida Avenue, Richmond, CA.  Some more details on the equipment.   The network identifiers (SSIDs) include ‘archive.org’ in their names, and the 2.4GHz ones are open with no password or encryption.  Thank you to Ralf Muehlen for setting up this system, and thank you to the City of Richmond for allowing an tower to be installed with no delay or hassle.

Onward to a Free and Fast Internet for All!

Press: SF Chronicle   SeattlePI

 

Posted in Announcements, News | 16 Comments

Trovebox adds support for Archive.org storage

Posted on April 18, 2013 by internetarchive

Photo storage and organization service Trovebox announced today that they added support for storing your photos at archive.org.  Or as they put it:

tbx-archive.org

 

 

Check out their announcement.  We’re excited to host their patrons’ photos and keep them safe.

Posted in Image Archive, News | Leave a comment

Site down some of Tuesday and Wednesday for Power Upgrade

Posted on April 15, 2013 by Ralf Muehlen

[Update:   Upgrade is done, we were offline twice, as we predicted (and are sorry about), but now we have twice the power.

New transformer for the Internet Archive Building.

New transformer for the Internet Archive Building.

Thank you PG&E, Ralf Muehlen, and the Archive engineers.]

This week, we are doubling the power coming into our primary data center so that we can archive and serve even more web pages, books, music and moving images. During those upgrades, there will be times when many of our web sites and services will not be available. Details below.

To keep the data safe, we will proactively shut down most of our services served from our primary data center. archive.org, openlibrary.org, iafcu.org and our blogs will be unavailable during the outages. The upgrades will happen over a two day period. We anticipate two prolonged outages, the first one from about 7am to 12noon PDT (14:00-19:00 UTC) on Tuesday, April 16. And the another one from 3pm to 7pm PDT (22:00-02:00 UTC) on Wednesday, April 17. Work might require additional outages between those two major ones.

During the outages, we’ll post updates to our @internetarchive twitter feed. Sorry for the inconvenience.

Update: To be on the safe side, we’ll expand Wednesday’s outage window from 2:15pm PDT to 7:15 PDT (21:15-02:15 UTC). For some of our services, the actual outages might be shorter.

Posted in News, Technical | 3 Comments

Archive of Historical Computer Software is here

Posted on April 14, 2013 by brewster

Thanks to Jason Scott, lots of deep collecting communities, and volunteers, Jason is announcing that the Internet Archive now hosts some very large software and computer documentation collections, maybe the largest overall host.

Yippie!

Now we all have to make it larger, more findable, and re-usable– please help, please donate money, time, anything– this is our history, lets write it well.

 

Posted in Announcements, Software Archive | 10 Comments

Celebrating 100 million tasks (uploading and modifying archive.org content)

Posted on April 12, 2013 by traceypooh

Just over 8-1/2 years ago, I wrote a multi-process daemon in PHP that we refer to as “catalogd”.  It runs 24 hours a day, 7 days a week, no rest!

It is in charge of uploading all content to our archive.org servers, and all changes to uploaded files.

We recently passed the 100 millionth “task” (upload or edit to an archive “item”).

After starting with a modest 100 or so tasks/day, we currently run nearly 100,000 tasks/day.  We’ve done some minor scaling, but of the most part, the little daemon has become our little daemon that could!

Here’s to the next 100 million tasks at archive.org!

-tracey

Posted in News, Technical | Tagged , , , , | 1 Comment

450,000 Early Journal Articles Now Available

Posted on April 11, 2013 by brewster

jstorlogoInternet Archive announces today the addition of over 450,000 journal articles from the JSTOR Early Journal Content collection. Early Journal Content is a selection of pre-1923 materials from more than 350 journals and includes articles in the arts and humanities, economics and politics, and mathematics and other sciences. This content was digitized by JSTOR and is freely available through jstor.org, and it can now also be accessed and downloaded via archive.org.

Screen Shot 2013-04-09 at 10.58.20 AMHeidi McGregor from JSTOR said, “We’re happy to work with the Internet Archive to broaden access to the JSTOR Early Journal Content even further, offering people the ability to use it alongside other Internet Archive held collections.”

Screen Shot 2013-04-09 at 12.15.43 PMAll 2 terabytes of the Early Journal Collection are available for bulk harvesting from the Internet Archive. Web search engines have been indexing the full-text contents of these materials already and, so far, people and robots have downloaded the articles over 400,000 times even before it has been announced. A data bundle including OCR text and metadata is also available from JSTOR’s Data for Research service for free downloading.

Posted in Announcements, Books Archive, News | 19 Comments

Open Call for tumblr Collaborators

Posted on April 8, 2013 by internetarchive

Screen Shot 2013-04-08 at 3.25.52 PMWhen it comes to collaborative culture, tumblr is where it’s at – and we’re ready to jump in. We’re not going to just redirect this blog, though, we’re opening up our tumblr URL to anyone interested in messing around with our content.

We’re looking at this as an opportunity to show the world some of the amazing stuff we’ve collected – over 10 petabytes of information just waiting to be juxtaposed, made into macros, remixed, glitched, written on, moshed, analyzed, sequenced and combined in ways we haven’t dreamed of.

We will be accepting 52 people. We’ll be here to offer support and guide them in their exploration with content and code, then we’ll feature their finished work for a week on the official tumblr. Each person’s residency will also be archived, of course. That’s what we do!

Check out http://internetarchive.tumblr.com for more details and an application form.

Posted in Cool items, News | 1 Comment

How the Internet Archive is having Great Time with Bitcoin

Posted on April 3, 2013 by brewster

bitcoinInternet_Archive_logo.ai

The Internet Archive jumped into bitcoin in 2011 after many people asked to donate bitcoins– a non-techy set up our wallet (good sign), and we got about $2,000 worth of donations that first year.  In 2012 we got about $6,000 worth– interesting.

I think of it as the local currency of the Internet. I figured out how to do it (then found an easier way), read the key paper, watched a good intro video, loved watching bitcoin in realtime avoid a financial crisis without a bailout.  This is an “anti-fragile” system that is getting debugged.     Feels like linux where exposing bugs are thought of a good thing rather than prosecuted as a crime.

Then the fun began!  We offered employees to get partially paid in our donated bitcoins– 1/3 said yes, now we are buying sushi and beer.  We were written up in Bitcoin Magazine, The Next Web, arstechnica.

Sushi for Bitcoins

Sushi for Bitcoins

 

We set up an honor based Bitcoin ATM to convert $ to Bitcoin and Bitcoin to $.   Many people now use it.   Hacker space at NY college following, maybe hacker space in SF.

We offer to take Bitcoin to see a movie about Anonymous at the Archive.   5 people pay with bitcoin.

Bitcoin is becoming a day-to-day currency of the Internet Archive with employees using it as a way to settle small debts, like for dinner.

If you want to try it then set up a “thin” bitcoin wallet on your phone or laptop, come to the Internet Archive, convert some money, then go out to lunch,  drink beer and celebrate.

Completely fun, and very very interesting.

 

 

 

 

Posted in Announcements, News | 44 Comments

Riding with the Bit Savers

Posted on March 12, 2013 by Jason Scott

Since 1997, a dedicated team of scanners and curators have been assembling a collection of historical computer and technology-related items. This collection, called BITSAVERS.ORG, contains tens of thousands of documents and software products dating back from the 1950s and into the 2000s. From the days of mainframes and electronic counting machines through the home computer revolution and the short lives and shorter support of various pieces of equipment, Bitsavers volunteers have been scanning industriously. There are piles of manuals and brochures, as well as guides and overviews, that have been cast aside in favor of the next big thing. Bitsavers has been working tirelessly to rescue these lost documents.

And now, they are mirrored on the Internet Archive.

Bulletin_618-1_Versatec_Spectrum_Jun1986_0000M-100-65_MDS-1101_Brochure_0000

Currently, over 23,000 individual manuals, books, memos, and guides are hosted on the Archive in the collection, automatically ported over from the Bitsavers mirrors.

Every week, a dozen or more new documents join the Bitsavers archive, from all reaches of technological history. Whether you want to browse the original manual for the Apple I or learn the benefits of a Sanders Associates 5700 Tape System, there’s something for every person interested in seeing where computing has come from.

Some other gems in the collection:

Whether for research, nostalgia, or interesting inspiration for artwork and writing, the millions of scanned pages in the Bitsavers collection are a click away from the collection page. Where possible, further sub-collections for companies like IBM, DEC and Control Data Corporation are also available.

A toast to this flood of computer history!

22-6275-0_Functional_Wiring_Principles_0015070-5199-01_4100F19_4690_Series_Color_Graphics_Copier_Interface_Instructions_Feb1985_0009EK-VT105-TM-001_VT105_Graphic_Terminal_Technical_Manual_Sep79_0168

 

Posted in Announcements, News | 3 Comments

UCLA Brings Light to the Undiscovered Country of Television News

Posted on March 8, 2013 by Roger Macdonald

The UCLA Library recently launched a remarkable broadcast news  research and education platform, Broadcast NewsScape.   The service is accessible online to users on the UCLA campus.  Platform managers hope to expand access throughout the UC system later this year.  NewsScape captures closed captioning, in a manner similar to our TV News Search & Borrow, to facilitate deep search and discovery of relevant segments of over 200,000 U.S. and international news program episodes.

BroadcastNewsScape1We are excited that the UCLA library has joined Vanderbilt University and the Internet Archive in offering tailored research and public interest access to television news.  These successful demonstrations of responsibly providing public benefit access to television news are helping to enrich conversations regarding mutual benefits among media and library stakeholders.

UCLA has a storied history in archiving television news, starting with the 1974 Senate Watergate hearings.  Between 1979 and 2003, UCLA recorded off-air more than 100,000 news programs, preserving and making them accessible in UCLA’s Film & Television Archive’s News and Public Affairs Collection  In 2005, Communication Studies department professors Francis F. Steen and Tim Groeling brought UCLA’s television news archiving into the digital age, recording direct to disks and, most transformationally, preserving available closed captioning.  Their collection has enabled researches to experiment with new digital processes for analyzing attributes of broadcast news.

NewsScape_infrastructureLast year, the UCLA Library started making provisions to take the digital news archive under its wing, devoting considerable server resources and relieving Francis and Tim from their 8-year labor of love maintaining their modest, sometimes cantankerous, hardware and ever-growing data stores.

Thanks to the leadership of associate university librarians Todd Grappone and Sharon Farb, the UCLA Library’s newly launched Broadcast NewsScape tool is welcoming scholars, educators and students from throughout the university to delve deeply and and derive new insights from the undiscovered country that is television news.

UCLA’s announcement: http://newsroom.ucla.edu/portal/ucla/ucla-library-launches-transformative-243873.aspx

Posted in News | Tagged , , , , , , , , , , , , | 4 Comments

Friday March 22nd Movie and Panel about Anonymous: We Are Legion

Posted on March 5, 2013 by cameron

Screen Shot 2013-03-06 at 10.46.56 AMPlease join filmmaker Brian Knappenberger, journalists Quinn Norton and Ryan Singel and lawyer Thomas Nolan for a showing of “We Are Legion: the Story of the Hacktivists.” (trailer) The documentary about the hacker/activist group ‘Anonymous.’

Recorded Video of the Panel

Friday, March 22nd:
6pm – Reception
7pm – Film
8:40pm – Panel discussion

  • Brian Knappenberger – Director, writer and producer of the Film.
  • Thomas J. Nolan – Attorney at Nolan, Armstrong and Barton, LLP, certified criminal law specialist one of Daily Journal’s 100 most influential lawyers in California.
  • Ryan Singel – Journalist, founder of Contextly, co-founder of the Threat Level blog, and previously at Wired magazine.
  • Quinn Norton – Journalist and blogger covering hacker culture, Anonymous, the Occupy movement, intellectual property, copyright issues and the Internet.

Location:
Internet Archive
300 Funston Avenue
San Francisco, CA

$5 bucks or 5 books or 0.10 bitcoins.

Please RSVP to help us keep a rough count.

Q’s: info@archive.org or 415 561 6767

See you there!

Posted in Announcements, Event, News | 15 Comments

Bitcoin <-> Cash Converter Box

Posted on March 5, 2013 by brewster

bitcoin_converter

Bitcoin to Cash Converter Box

 

bitcoin arrowgold_dollar  gold_dollararrowbitcoin

 

 

(Please leave this page visible on the computer next to the cash box).  To help us try out bitcoins, I am putting up $200 ($100 cash and $100 worth of bitcoins) to make an honor-based converter box to be available at the Internet Archive Friday lunches.    Please donate a $1 conversion fee for each transaction to help cover loss and mistakes.  If this works, then maybe other offices or hacker spaces will do this.   Please leave this page visible after you finish.

Convert your bitcoins into dollars:

 

bitcoin arrowgold_dollar

 

Calculate the conversion (opens in new tab) and check to make sure we have enough cash and correct change in the cashbox (current limit is $100).  If not, email me.

Then use your bitcoin client to send address: 1Pt9TRJKeAW61aR1ELQpUZKdMaYXzkCTrn, bitcoin-qrcode (you can send a skype from this machine with the address or use this webpage on your own machine).      Take your dollars from the cash box.  Please leave $1 for each transaction so it will cover for loss or mistakes.   Please leave this page visible after you finish.

 

Convert your dollars into bitcoins:

gold_dollararrowbitcoin

Calculate the conversion (opens in new tab) amount for the dollars you want to convert and make sure it is under $100 (our current limit).  Please subtract $1 from the amount you want to covert as a conversion fee to cover loss or mistakes.    For more than $100 maybe think of using coinbase.

Then use the Bitcoin-qt application on the computer (right hand one of the displays in the Internet Archive library).   Make sure it has enough coins, please email me if not.   Skype your bitcoin address to this machine (bitcoinconverter).  Use the Send Coins button on the application window, and send yourself the coins.

Please leave this page visible after you finish.

 

If you want to do this, this is the cashbox we got:

cashbox

 

 

 

 

 

 

Posted in Announcements | 22 Comments

Idea for a WordPress Plugin for Low-End Bitcoin Storefront

Posted on February 25, 2013 by brewster

wordpress-logo-notext-rgbBitcoin

The Internet Archive would like to take donations and sell some swag for bitcoins via our wordpress blog.   There are a number of quality third-party plug-ins we could use, but we are dreaming of a simple system that has no third parties involved.

Goals:

  • Minimal engineering time to set up,
  • No third party involvement if possible for privacy and simplicity reasons,
  • Straight forward process for our front-office staff to fulfill and track orders,
  • Easy to add a new offering,
  • A plug-in that others could use to add this to their blog,
  • Open source so others could build on it (we use agpl-3).

Our idea is to have a form we can stick at the bottom of any page asking for shipping information or whatever, and supplying a Bitcoin address and QR code for them to send to, and a “submit button”.     The user would then see an order confirmation page, get an email confirmation.

The website would get an entry in a tab delimited file written to our wordpress world and an email alert being sent that a new order came in.     The tab delimited file could be used by a front-office person by importing into a spreadsheet– super simple.

The bitcoin addresses would rotate through a set of addresses, maybe 1000 of them, that we generated from bitcoin-qt and then put in a file on our WordPress world.   This way we can check to see that the address that someone said they would send to got those coins at about the time they said they ordered.    We realize this has potential problems (like if we get a flood of over 1000 orders in short order, or if someone hacked our wordpress and substituted different addresses), but it has the advantage that our wallet would be on a front-office person’s computer like it is now.

Any ideas on this, or anyone up for helping do it?

 

 

Posted in News | 2 Comments

Employees to be Paid in Bitcoin: Please Donate

Posted on February 21, 2013 by brewster

(Press about this:  Bitcoin Magazine, The Next Web, arstechnica)

As Bitcoin is becoming the “local currency” of the Internet, the Internet Archive would like to help support it and use it.    In the last 2 years, over 300 people have graciously donated bitcoins to the Internet Archive.

Some Internet Archive employees have elected to receive some of their pay in Bitcoin in April.     If this is successful we hope to make it a continuing option.

Please help.   If you would be up for helping fund our employees, please consider donating to the Internet Archive.   Our bitcoin address is: 17gN64BPHtxi4mEM3qWrxdwhieUvRq8R2r  but other forms of money are also welcome.

 

Posted in Announcements | 53 Comments