Welcome back to Computer Science E-1!

Lecture 7: Multimedia

November 1, 2006

Lecturer: David J. Malan

This is lecture 7 on multimedia; let's start with a snippet of multimedia

Macs VS. PC VOTW

Each week, teaching fellows along with Chris Thayer produce and edit videos of the week for the course

Surf on over to the course's website; the front page has changed. Two ways to access the course:

Through the traditional way

Apple's iTunes U

Have you noticed the Thanksgiving theme or the fluttering objects?

If you go to Videos of the Weeks and scroll down, you can see all the videos of the weeks

Not required watching for the course, but bite size lectures on various topics

You can watch them in 2 different ways, in Quicktime or embedded right in your browser as a Flash version.

David: "I do love the Macs vs PC video, even though it does take a strange turn towards the end"

Survey results: Some of you have felt a bit overwhelmed by all the content on the website. We admit that the menu has been a bit long, so we have trimmed down the list a bit. If you haven't noticed a thing yet, please do visit

Two comments:

What do you think of lectures specifically? "I have yet to nod off"

What do you like about lecture specifically? "I enjoy free food and candy"

Let's forge ahead with our lecture on multimedia.

Multimedia: What is it? (-> mark indicates student response)

->Stuff that's not text

->Composition of audio, video, and text, in a more animated/dynamic way

What kinds of files, content?

-> JPEG, sound

Where else have you seen examples of multimedia in the course?

-> video, Powerpoint slides

Especially more stylistic Powerpoint slides with animation

-> Skype, Google Earth

Forms of multimedia in that they are interactive

But no steadfast definition, a combination of media

Give an example of a form of multimedia that you may have seen today at work

-> different SWF (Shockwave files)

-> e-vite <-- okay, has some animation

-> simple webpage

At least some graphic on most webpages. The graphics that you interact with on the WWW are a form of multimedia. In what format are those images?

-> JPEGS, GIFs

By the end of tonight, you'll be able to talk about some of the differences between the 2 formats.

GIF: one of the most common file formats on the web. If you see a file called david.gif, it appears to a file in the format of GIF, just like resume.doc is a Word document. Just another layout of 1s and 0s in a specific pattern in a graphical format. Capable of representing 256 colors, animation, and transparency.

Format: standard layout of 1's and 0's determined by software companies

Any graphic on the computer is laid out in a rectangle; for example 200 pixels tall and 100 pixels wide.

What is a pixel? Who's seen a pixel today?

A pixel is a measurement, it's a dot on your screen. If you put your eye very close up to your screen, you can see many tiny squares or dots.

What are some common resolutions for a monitor?

800x600

1024x768

1600x1200

While doing one of the problem sets, you may have seen the resolutions in relation to monitors. If you use Windows, you can go to Start -> Control Panel -> Display Properties -> Settings

With a projector, some resolutions will not be supported. So you may have to fiddle around

Your monitor's resolution is basically your real estate for display. Video and graphics have set resolutions; if bigger than your screen, it will not display properly because you do not have enough pixels.

Each dot can represent a color.

Animated GIFs: You may have seen some websites with little moving graphics. They typically use animated GIFS, which are basically little 'flipbooks' that keep repeating themselves.

Transparency: Say you want to put an animated puppy on a blue sky. It would look pretty ugly if your animated puppy had to have a white rectangle behind it. Thus, with a GIF, you can specify that a certain part of the image's 'rectangle' is transparent and anything underneath the image can show through (on a website).

But what if part of your animation is the same color as the part you want to remove? Simply make the part you want to get rid of a color that is not in the main object.

Next familiar format: JPEG

JPEG: Good for digital photographs; may have seen them while uploading images from your digital camera to your computer. Capable of millions of colors, which allow you to represent a larger range of colors. Incapable of animations.

if you attend this week's Section, you can learn how to use software to do animations and transparencies, such as Adobe Photoshop.

What are the specs on your camera?

-> 7 megapixels (MP), 5 MP...

Why did you buy so many MP? Why do you want so many?

--> sharper images, more depth, you can enlarge, more detail

i.e. if you took a picture of the Grand Canyon and your camera only took 200 x 100 pixels, if you tried to print the photo, it would be hard for all the dots to fill a 5x7 or 4x6 space. It would be very blurry or splotchy because each dot is a really big square dot.

Or if you look at a newspaper and look at an image on the page. The comics, for example, are in color and printed as dots. From a distance, they look fine, but look at it up close and it doesn't look so good.

When cameras first became popular, you could get maybe a 1 or 2.1 MP camera - but what does that mean?

-> It means what you just said

When the Sonys and Canons say "x MP", it's the height multiplied by the width, so the more MP, the more area, the more pixels to print with, allowing for more zooming and cropping options.

So if you're in the market for a digital camera, more MP, it gives you more quality and flexibility, more dots to play with.

-> When you have more MP, is it more compressed or is it more area? Is it a wider lens?

It is more area. Say I try to take a picture of the teaching fellows, if I only have really low-res, then less dots are devoted per subject. But with more MP, more dots can be used to represent each subject.

-> Is it possible to compress the image more in some areas than others?

Theoretically, yes, however in actuality, the compression stays consistent all across the image. In fact, JPEGs not only specify how 1s and 0s are laid out, they can determine levels of compressions.

PIZZA BREAK!

Alright, we are back! In case you are wondering, the pizza is a thank you present from HILR (Harvard Institute for Learning in Retirement) for the workshop on how to use the Internet which took place last Sunday (Oct. 29). Several E-1 students participated as teaching assistants.

If an image is in a GIF or JPEG format, they are bitmap formats, which means they are composed of pixels. Good thing: each pixel can be represented by a different color. Bad thing: if you zoom in too far, they get blurry, thus limiting how big you can blow things up.

i.e., in crime shows, they ask to zoom in on a video - well, it is impossible to zoom in forever (despite claims of mathematical interpolation - smoothing out of elements) because there is only so much information. In courts, this is interesting because interpolation is a form of computer enhancement.

Well, if GIFS and JPEGS are bitmaps, are there formats that can be zoomed in, etc without getting blurry? Yes - vectors, mathematical representations of graphics.

For example - a circle of dots. Wouldn't it be better to represent a smooth perfect circle with a formula? Then you can resize indefinitely.

So with a picture of cheese - break each element down into polygons, scale each element as is convenient

SWF: file format for animations that is not just graphical - can have audio too. Also is interactive and VECTOR-BASED. A lot of "flashy" websites use .swf files to increase interactivity.

Example: 4 Swedish horses movie - very small animation on a 1024x768 screen (popular resolution for today). Well, what if we zoom in? Will they get square and blotchy? No! They actually appear sharper because there are more dots representing the mathematical formulas that make up the images. And they are a capella horses too! (link forthcoming)

Mathematics are used to make graphics appear clear. Math is also used in other areas - such as games. No longer just bitmaps graphics like Mario or Donkey Kong, today's games can represent 3D shapes and movement using polygons.

Also each motion, if represented flipbook style, would take up a lot of information. So using math like linear algebra, your CPU and video cards can calculate on the fly different actions of your characters.

If you click on the course's website, there is a set of retro games in Flash (SWF) format. Asteroids, Donkey Kong, Pong, etc.

Volunteers come down to play Frogger! (Heather)

So what is the lesson taken away from playing Frogger?

It is definitely bitmapped, the result of putting many colored dots on the screen

Very flat despite all the animations, very jarring, not smooth slides

Dan goes up with his laptop

Both GIF and JPEG are compressed - what does that mean?

--> to make it smaller?

Whatever happened to PNGs? Coming later

Well what does it mean to compress? I could compress a Word document by highlighting all the words and deleting them. But that's extreme. Basically, to make something smaller, you will need to lose information. But how is it decided what is "lost"?

Lossy - compression by loss of information

i.e. the German and French flags - both have only 3 colors, 300x200 pixels (60,000 pixels total). Is it possible to represent the flags in the same clarity using fewer pixels? We have to keep the size, shape, color...

-> can't you just make the flags "smaller" (make the actual image size smaller, so that you need less pixels)

True, but that would require deleting information. So, say we have to keep the flag that size?

-> Represent each pixel as a number and place them in a larger matrix

Closer - you can store one column of pixels (a set of black, red, and yellow pixels), then multiply by 300. Basically, there is a lot of redundant information - 3 blocks of colors.

-> Retain as much information that you need in one column, then multiply it by a number to fill across the rows.

GIF is lossless - it does not lose information with manipulation

So, you can say "this pixel is black, multiply by 66, etc. until you have your column, then repeat the column across the space (horizontally) 300 times!"

Does France's flag compress worse than Germany's? Yes, because the bands are vertical, they are not contiguous. The calculations compression goes from left to right, so you have to store 3 different columns instead of 1.

Is it possible to have hybrids? Yes, but generally you will only see GIFs, JPGs, and occasionally PNGs on the web.

On the website, for the problem sets, I noticed a new format, RTF? Is this new? No, it's been around for awhile, not created by Microsoft, but has a lot of the same features as a Word document. Not a multimedia file format.

Can we see all the other handouts in RTF format? Sure, but Adobe Reader allows you to highlight.

Whatever happened to PNGs? (Portable Network Graphics) It's like a hybrid of GIFs and JPEGs in that it is lossless but represents millions of colors. It is not proprietary and was actually created in response to the fuss surrounding GIFs. Go check the computer dictionary for more technical details.

So for a picture of an apple on a blue background, all you need to keep is the apple image, plus a column of blue because the computer can be told to fill in anything that is white with blue.

Video is in vogue these days, especially with bandwidth and diskspace increases, Youtube and Google Video are gaining popularity. Youtube displays Flash videos, which allows you to view videos without special software because it can be embedded inside the browser. Well not exactly - you do need special players (software), but according to Adobe, approximately 97.2% of users have those players already installed on computers courtesy of manufacturers. More and more browsers also automatically install the plug-ins, unlike the in the past.

However, something like Quicktime requires a special external player. Unlike Flash, it does not come with every Windows PC these days. You would have to download it specially. These days, it comes with iTunes from Apple.

So we pull up a video - we need a volunteer who is willing to be on camera and on camera. (Built in webcam on the MacBook Pro is capturing the subject, which is projected onto the screen and captured by the regular video cameras. It is being re-interpreted by a piece of free software that comes with newer Macs)

Example of how computers are now powerful enough to render immediately different effects in video (i.e. pixelations, distortion, fisheye lens, bulge, etc. - like funhouse mirrors). Can take photos of the subject in that effect in real time without post-production.

-> Can you scan in a piece of artwork and have this software do the same renderings?

Yes, all you have to do is hold the artwork in front of the camera and it will take a picture the same way it captures live subjects. You cannot layer effects using this program though, which is available with all newer Macs that come with built in webcams.

So Problem Set 5 is hands on and graphically oriented - On the course's website, there is currently a big turkey in the upper left hand corner. If you click on the 'banner', you can change the images, which have been created by former E-1 students.

Hands down defacto program for graphics work is Adobe Photoshop. Though we try to be platform and software independent, Adobe Photoshop is available as a 30 day free trial or through Harvard using your student id.

This week's section will be hand-holding tutorials on how to use Adobe Photoshop, which will help you with the activities in the problem set. Afterwards the submissions will be put up on the website in the banner for some time. Also, for extra credit, you can make a candidate design for the course's annual mousepad, which is voted on by the student body. Everyone gets one as a souvenir.

Bring your attention to the blue handout: the description for the final project.

Despite not having had the website lecture, you can still do the first part of the project - deciding what you want to do for your website. You also decide what it will be called - it can be anything you like, so long as it has not been already bought. All TLDs are game - you do not need to have a .com. You do not need to buy a domain, but E-1 will host your website for free through May, about a GB of space. The sky is the limit!

The first part of the assignment is due at the end of November, technical details will be covered more in depth in the next lecture.

So let's go back to where we began - the Mac VS. PC video.

It is a Quicktime format - which was developed by Apple. Comes automatically with Macs, but with PCs, you have to specially download it and then load the .MOV files. The gray box that appears is the Quicktime player.

If you go up to the File menu, you can ask to show the movie info. What is the resolution?

-> 594x446 - current size

-> 320x240 - native size, the actual resolution

Why does that sound familiar? Multiply both dimensions by 2, you get 640x480, a standard resolution. So, these videos are quite small, thus if you blow the videos up greatly, you can start to see that they get a little blurry. What is interesting about when we freeze frame an image of Rei?

You can see that his outline is fuzzy, not very sharp - at least not as sharp as the Swedish horses. Why? Because likely the file format being used is JPEG.

Within videos, you have frames - like a TV show or an old film or like a flip book, but with a lot of frames. How many frames are you being shown when watching TV or a movie? Approximately 24-30 frames per second. At that point your eyes can't differentiate between frames, creating the appearance of motion.

Quicktime (.MOV)

Flash (.FLV)

RealVideo (.rm, .ram, etc.) <-- course lectures are in this format, developed by a company called Real

What is interesting about the course's videos? You can get a split screen format that shows the PowerPoint presentation in synch with the video (courtesy of a special timing software used by the video folks)

RealPlayer has a lot of details/features/stuff that may be unnecessary, so you can watch RealVideos with RealAlternative which just plays the video.

When you start playing a RealVideo, those videos can be about 200-400 MB big. They usually play immediately, with some buffering - is your computer really that fast? Probably not. So what's going on? It's streaming! And all three above formats - .mov, .flv, .rm - can be streamed, which means they can start playing without the entire file being downloaded first. They do this by downloading a little bit, then once it has a "buffer" of 5 seconds or so, the player starts playing, while downloading about 5 seconds ahead. But sometimes the video stutters or freezes for a few seconds - why?

-> Because of RAM?

Not quite...the movie is playing faster than it is being downloaded, meaning that the buffer was just not long enough. This can be the result of your internet connection not moving fast enough or getting interrupted (packets are dropped), so the data has to be retranslated, meaning that you have to wait.

Most of these videos are pre-recorded - but what if you are trying to watch/listen to something live? You can do so using programs like iTunes, which stream (an available feed of bits) in the data. When you tune in, you "jump" in at whatever point that the stream has reached, like with radio stations. Some local stations may stream out via the Internet. An interesting note is that there can be 3 versions of the streams, of different bitrates. The higher bitrates can lead to higher quality of sound, but it literally means that that many bits are being downloaded per second (i.e 128 kb per second). However, with a slower connection, you might want to tune in to a lower bit rate (ie 96 kb per second) since if you buffer, you will information. So choosing a lower bitrate can keep the feed going more smoothly. (Compression by degradation of quality) (Radio station featured: Groove Salad)

Sometimes the players will buffer the streams anyways, like the shockguards on portable CD players.

Streaming technology lets you watch/hear something in real time. But rather than buffer, it may just drop you down in quality to keep the stream going or even drop bits, hence missing sections of data in favor of moving forward.