FOSS Community

Learn Open Source Database Tools from Stanford for Free

I recently finished Stanford’s excellent free on-line course Introduction to Databases with Jennifer Widom. The course is a broad survey of database technology including XML, Relational Database Management Systems (RDBMS) from many angles (SQL forms the centerpiece of the course), OLAP (OnLine Analytical Processing) and NoSQL.

I was very impressed with the breadth of Widom’s approach to the subject: it was a major reason I decided to spend time on the course. Another strength is its nuts-n-bolts approach: some theoretical topics are covered but for the most part this is a course for practitioners. Finally, I particularly appreciated the extensive use of FOSS (Free and Open Source Software) in the course.

Why study databases? I will merely say that data is a core tool pervading the information resources of modern civilization. Databases are where data is housed. For example, the data constituting this blog is stored in a database and the same can be said for much (if not all) of the Internet. Databases are a profoundly vital, big picture subject.

Widom’s course is still open for enrollment in “archival mode” meaning you can watch the videos, work through the exercises, quizzes and exams, and track your progress, but the deadlines have expired and no more “Statements of Accomplishment” will be awarded (at least until the course is offered again). To complete the simple enrollment go to db-class.org and start learning about databases with FOSS today!

Although the course is broadly useful for anyone wanting to learn the basics of databases from a broad perspective, I found it to be particularly good for learning about the FOSS tools that can support database systems. So let’s start there.

FOSS Tools Covered

For traditional RDBMS, the course uses PostgreSQL, SQLite, and MySQL. Widom mentions some limitations of each database (DB) in regards to the SQL (Structured Query Language) standard including important distinctions about using each system with triggers, transactions (PostgreSQL has the best support), Views (MySQL uses updatable views whereas PostgreSQL & SQLite use triggers to modify views), recursion (only PostgreSQL and only in newer versions), and OLAP (only MySQL supports with rollup).

On the XML side, I learned xmllint and SAXON for XML validation, querying and transformation. The course covers the basics of DTDs, XML Schema, XPath, XQuery and XSLT. I used xmllint, saxonb-xquery, and saxonb-xslt to work through the exercises (the searchable Q&A forum provides usage details).

Finally, for NoSQL there are two videos which survey the state of the art. There is some depth on the Map Reduce framework provided by Hadoop. Several other FOSS systems are briefly discussed: Cassandra, Voldemort, CouchDB, MongoDB, and more. The NoSQL portion of the course is a good overview of the technology, but there are no exercises and hence little depth of a concrete nature.

From a FOSS perspective, the course is exquisite: FOSS utilities were front and center for the duration and some guidance in using these tools is provided. Help was readily available: I answered a few questions in the Q&A forum to help people overcome hurdles and I used its search feature to overcome some of my own. In sum, studying this course will give you the lay of the land for FOSS database technology including some advice about the limitations and strengths of its best database tools.

Stanford’s innovative platform for free on-line video courses

I am a big fan of so-called Open Educational Resources (OER) including free on-line video courses. Stanford’s Databases course is the 12th I’ve completed, but only the second in which I did a “deep dive” by reinforcing learning with exercises, quizzes and exams. In general, I use OER video courses as edutainment as I usually find the extra work too time-consuming: my goal is to broadly understand how the world works, not to build expertise in every subject I study! So, conceptually, I prefer the traditional form of video courses pioneered by MIT’s OpenCourseWare which in contrast with Stanford’s new approach might be called archived courses. Archived courses make the material available without (m)any social tools. So, working through the materials in traditional OER courses usually requires extra self-discipline and commitment (unless you just watch the videos for fun as I often do).

Stanford’s OER system, online at coursera.org, builds on the basic idea of OER video courses by adding deadlines, interactive feedback from automatically evaluated work, and some, including the Databases course, offer the ability to earn a “Statement of Accomplishment” for demonstrating basic proficiency. It is precisely these social enhancements that makes Stanford’s initiative so noteworthy. Together these social tools provide a shared experience with a clear set of tasks for a cohort of students working through the course at the same time.

The extra interactivity and the focus of deadlines give the Stanford approach to OER a special excitement and sense of goal accomplishment which is absent in archived courses. Even though I prefer the archived courses whose videos can be more entertaining than Stanford’s tutorial-focused approach, I have to admit I was enthralled by the deadlines: they kept me focused. It should be emphasized that Stanford’s courses like the more traditional OER archival courses can be pursued at a pace that suits your time and interest: there’s no imperative to follow the deadlines or earn kudos for accomplishments.

How I used the course materials

Although I have been using databases professionally for many years, I had not read nor studied the subject in any depth previously. I decided to take this opportunity for a deep dive. Widom’s Databases course includes a simple enrollment process, tutorial-style videos (for download or in a browser with Flash support), automatically graded exercises, quizzes, and exams each with hard deadlines, a Course Materials section with many goodies, Optional Exercises, a FAQ, a Q&A forum, and a “Statement of Accomplishment” from the instructor if you completed a substantial portion of the coursework by the deadline (6,513 of the 91,000 students enrolled in the Fall 2011 Databases course earned one; here is mine).

First, I watched each video twice taking detailed handwritten notes on the second viewing (22 pages worth!). I then checked the Flash version of the videos which often included inline questions that were very useful (Stanford’s Flash video viewer is the best I’ve seen: it even supports speeding up the video by 1.2 or 1.5 times while automatically adjusting the pitch!).

Then I worked through the quizzes and exercises. One nice feature of both was that you could attempt them many times. Different variants were provided to many of the quiz questions to make it harder to apply a blind trial and error approach and you can continue to work on them after getting them all correct (which might be useful as a way to practice for the exams or to see if you remember anything of the course when you check back in 1 or 10 years). I found the quizzes and exams to be very challenging and not so rewarding. Of course some mastery is required and acquired from the quizzes and for other learning styles they may prove more valuable than they were for me: judge for yourself.

The course included many supplementary exercises to provide extra practice. I used them for Relational Algebra and they were very helpful. However due to time constraints, I was unable to use them further. Most of my time was spent working through the exercises. I did all the exercises in an xterm window running sqlite, psql, xmllint, saxonb-xquery, or saxonb-xslt and pasting the results into the query workbench. This allowed me to really experience the FOSS tools “in the wild” which gave me a strong sense of their ins and outs. For me the interactive exercises were fantastic: they really helped me learn the material by directly engaging my problem solving faculty. They were like a real project with deadlines! Although I occasionally got ruffled with some of the difficult ones, they were engaging and fun!

Although the course web site is very simple and well-designed, it was still possible to have difficulty finding some of the gems provided for the students. For example, it took me awhile to find the code used in the demos (which was extremely useful by the way): the Course Materials section of the site has all the goodies you need but you have to mouse over the icons to see treasures that appear hidden at first (remember to right click to download). Also look carefully at the prescripts or postscripts affixed to some sections: more treasures.

The Q&A forum was helpful for finding things that were not at first apparent. A couple of times I scoured the Internet or Wikipedia looking for other angles on the material to understand a point I was struggling with. All work for the class is “open book”, so I only prepared for the exams by simply reviewing my notes.

Advice for students

I recommend taking the course now even though the deadlines have expired. Feel free to skip any part of the course that your interests, time constraints, and patience warrant. If the course is ever offered again, you will already know much of the material which should help you earn a “Statement of Accomplishment”. If not, you will better understand a broadly useful, important and interesting subject.

If you want to do a “lite” version of the course, I recommend skipping the quizzes and exams. In addition, many of the topics can be skipped if you are short on time or find them uninteresting. To her credit Jennifer Widom recommended as much in her screen side chats which provided a wonderful human dimension to the course. Although some of the material is cumulative, there are several parts of the course for which skipping is a real option. For example, the Relational Algebra (I thoroughly enjoyed doing those exercises!!!) and the Relational Design Theory topics are, I think, less important especially if you just want to acquire basic DBA skills. I deal with enough XML, that I found that part of the course extremely useful, but I can imagine someone who just needs SQL might skip those parts. This is a free course: you can be creative in how you use the materials so that you get what you want out of it!

I recommend doing the exercises associated with each topic (some did not have exercises). Some of them are quite challenging and some took me quite a bit of time. If necessary, skip some of the harder ones. If time is really pressing, just watch the videos that particularly interest you: remember this is a free resource: you can tailor your work on the course in whatever way suits your interests and time.

I did not buy nor borrow a textbook for the course (I was very impressed that Widom prepared reading assignments for four separate texts in the Course Materials sections of the site. Wow, that must have been a lot of work!). Having been through the course, I think a text is unnecessary for most students. You may find a few topics that are hard for you or for which the videos were insufficient to master the material. Since textbooks are more comprehensive and more detailed, they could help fill in the gaps. In particular diligent students may want a text. I prefer to learn iteratively, that is, I would prefer a shallow course today and another later that goes in more depth (I might even prefer to take two lite courses to build my mastery of a subject by degrees). But if you want a more complete experience now, then by all means get a textbook and dig in!

Other OER Database Resources

In addition to the OER resources below, I found occasion to reference Widom’s course site for Stanford students in the allied CS145 Introduction to Databases and her colleague Jeff Ullman’s offering of CS145 from Autumn 2002. MIT OCW has 6.814/6.830 on Database Systems which looks a bit too advanced for an introductory course and MIT OCW has 1.264J/ESD.264J Database, Internet, and Systems Integration Technologies which I found useful to supplement Widom’s course (especially for Relational Design Theory). Finally, the Indian Institutes of Technology (IIT) has a complete video course (with 43 videos totaling 40 hours and 17 minutes) on Database Management Systems or watch its YouTube playlist (I did not look at these videos, but I’ve seen other IIT material and they are usually very informative and accurate).

Conclusion

Stanford’s exciting new system for on-line courses is remarkable in its use of social tools to engage students. This is a boon to the OER movement! In today’s rapidly changing world, refreshing and expanding one’s skills is essential to apprehending the needs and opportunities that abound if you are curious enough!

I’m hooked! Although I still love and will continue to use archived courses, I will be checking to see if Stanford has any courses of interest regularly from now on! I’ve signed up for several of Stanford’s offerings (there are 16 of them!) for Winter term with the intention of “dropping” or doing a “shallow dive” (maybe just watch a few videos and do some exercises as interest permits). But I am eying the Model Thinking course for a possible deep dive. To see the full list of offerings go to Class Central: Summary of Stanford’s online course offerings and plan your learning for the Winter term which starts next Monday, January 23rd!

Stanford’s Fall 2011 edition of Introduction to Databases was a great course! Kudos to Jennifer Widom and everyone at Stanford who made this possible.

If you haven’t started or finished it yet, head over to db-class.org now and get to it! It is one of the best resources on the Internet for learning about databases. Moreover, it includes the special benefit of covering a broad range of important FOSS database tools.

Other Reviews of Widom’s Databases Course

Posted by CJ Fearnley in FOSS Community, 5 comments

Elizabeth Krumbach Keynoting at Fosscon in Philadelphia Saturday 23 July 2011

LinuxForce Systems Administrator, Elizabeth Krumbach will deliver the keynote address this Saturday, 23 July 2011 at FOSSCON. Her talk entitled “Make a Difference for Millions: Getting Involved with FOSS” will help attendees better understand how to contribute to the greater good by participating more actively in FOSS (Free and Open Source Software).

FOSSCON will be held at Basekamp, 723 Chestnut street, 2nd floor; Philadelphia, PA. The doors open at 8 AM on Saturday. Elizabeth’s talk starts at 10 AM. The other talks follow her keynote. A listing of speakers at FOSSCON and the schedule is on-line here. Register to attend FOSSCON for free (no charge!) to learn about the excitement of FOSS in Philadelphia!

I am looking forward to the event. I hope to see you there!

LinuxForce is a sponsor of FOSSCON.

Posted by CJ Fearnley in Conference, FOSS Community, News, 0 comments

Attending the Ubuntu Developer Summit in Budapest

On Saturday, May 7th, I’ll be taking a flight out to Budapest, Hungary to attend the week-long Ubuntu Developer Summit (UDS) as the kick-off event to the development of the next Ubuntu release, 11.10 (code name Oneiric Ocelot) coming out in October 2011.

The Ubuntu Developer Summit is the seminal Ubuntu event in which we define the focus and plans for our up-coming version of Ubuntu. The event pulls together Canonical engineers, community members, partners, ISVs, upstreams and more into an environment focused on discussion and planning.

My role at these summits as an Ubuntu Community Council member tends to be on community work, which includes recruitment and retention of volunteers to the Ubuntu community. I will also attend sessions related to upstream collaboration; most worthy of note are the collaboration sessions related to Debian as my primary development interest remains there. Debian is the parent distribution of Ubuntu, which LinuxForce almost exclusively deploys to our customers.

This will be my third time attending a UDS. I’m excited to see what I will learn, from the possibilities for the next release to the new ideas I will be able to apply in my day-to-day work. So much comes from such in-person collaborations with fellow contributors.

Posted by Elizabeth Krumbach in Conference, Development, FOSS Community, News, Ubuntu, 0 comments

VIDEO: Elizabeth Krumbach on the SDForum Panel Discussion on Women and Open Source

On March 31, LinuxForce’s Elizabeth Krumbach participated in the SDForum panel discussion on “Tech Women: Women and Open Source”. There was a wide ranging discussion starting with a basic introduction to “open source” and how to get involved in open source. Other issues covered included special issues with the involvement of women, mentoring, business, and entrepreneurship. The Google Open Source Blog also reviewed the session.

Video of the panel discussion is online. Part 1 is 52 minutes.

Part 2 is 21 minutes.

Posted by CJ Fearnley in FOSS Community, News, 0 comments

Slides for my talk on “Automating X11 Keystrokes”

X11 is the graphical user interface most widely used on Linux operating systems. My slides and video demo for a short talk given at the Philadelphia area Linux Users Group (PLUG) on March 2nd are on-line. The slides briefly cover xrandr (which can also be used to set the screen resolution), xset, xwd / xwud, xdotool, and xautomation including xte. You can get the slides and watch the video at my page on Automating X11 Keystrokes.

Posted by CJ Fearnley in FOSS Community, News, 1 comment

Finding Help in Ubuntu (SCALE, Feb 25-7 in LA); Ubuntu Diversity; Debian Squeeze AKA 6.0

Here are some news items:

• Elizabeth Krumbach will give a talk on “Finding Help in Ubuntu” at UbuCon in Los Angeles

Tomorrow, February 25th at 9AM, LinuxForce Remote Responder Administrator Elizabeth Krumbach will give the opening talk at UbuCon at the Hilton Los Angeles Airport Hotel. Her talk is entitled Finding Help in Ubuntu. UbuCon is part of SCALE: Southern California Linux Expo which runs 2/25-2/27 2011.

• Elizabeth Krumbach interviewed in Linux Pro Magazine

Linux Pro Magazine interviewed LinuxForce Remote Responder Administrator Elizabeth Krumbach in an article on Ubuntu Increasing Its Diversity. Elizabeth is helping with the effort to increase diversity at the Ubuntu Developer Summit in Budapest in May.

• A new stable version of Debian known as Squeeze or 6.0 was released

The Debian project announced the release of Squeeze on 6 February 2011. eWeek reviewed the new release. We have already upgraded 3 systems and installed several new systems running Debian Squeeze. It is very reliable as we’ve come to expect from Debian.

In addition, the Debian web sites received a new design, a new “look”, that was released together with Squeeze. Check it out at http://www.debian.org.

Posted by CJ Fearnley in Conference, FOSS Community, News, 0 comments

Beyond the Cloud: The Comprehensive Flexibility of FOSS May Bring Clearer Skies

Last week’s InformationWeek has a good article on cloud computing, Cloud ROI: A Grounded View.  It seems that even with all the hype (or because of it?) most are not “running blindly” to adopt “the cloud”.  I must admit the cloud metaphor has a powerful poetic charm to it.  That is probably why it has grabbed the attention of so many over the past few years. Everything in our world is ephemeral, so there is an aptness to the concept of a “cloud”. Moreover, I too like and use cloud analogies. But I am now looking for clearer skies!  Here is a short list of my gripes about "the cloud":

  • What does “cloud computing” mean? It isn’t at all clear! Here is some data: CIO magazine cites a Forrester report that says "the number one challenge in cloud computing today is determining what it really is". CIO also reported on a McKinsey study that "found 22 separate definitions of cloud computing"! And that leads to my second point:
  • The word “cloud” is so … vacuous and amorphous …  “A cloud:  it looks like Zeus!” only to transform in shape before your very eyes “Wait, now it looks like Aphrodite!” … and then its gone!  Is this the kind of model people should entrust with their business data?  It has no structural stability:  inherently:  it is just rapidly moving gases … far out of reach … away in the sky. What kind of business model is that?
  • Although RADLab (Reliable Adaptive Distributed Systems Laboratory) has put out some interesting papers, I was a little surprised when I read their acknowledgements in the CACM article A View of Cloud Computing.  It reads like a who’s who in cloud computing: Amazon, Google, Microsoft, Sun, eBay, Cloudera, Facebook, and so on. The original Berkeley paper has a shorter list of cloud companies funding their work. I’m sure they are maintaining their academic integrity, but it does show that they are not wholly independent. Remember what Kitty Foyle said:

    I’ve taught myself a lesson, or I hope I have: when I find myself thinking something I stop a minute and ask myself, Now who had it all figured out beforehand that was the way they wanted me to think?
    — From Christopher Morley’s novel “Kitty Foyle

  • Although the Berkeley papers raise a number of very interesting issues, none of them requires vacuous meaningless jargon to further obscure the subtleties and complexities of emerging technology trends! So my final gripe is that the name “cloud” tends to obscure what is really important even when I agree with “the cloud thinkers”.

Perhaps the most important issue “the cloud people” are missing is what might be called comprehensive flexibility. As a user of software technology, I want my computing functionality everywhere … in every imaginable format. For example, I’d like to be able to use the software that I’ve invested the time to learn to be available on my desktop (32-bit, 64-bit, Mac, Windows, or Linux), and I’d like it to work whether the Net is working or not, on my cell phone and other portable devices (again with network or not), in the data center (clustered or not), in the WAN (Wide Area Network, note that the Internet is our shared, global WAN), perhaps distributed among several hosting providers, and perhaps even provided by “utilities” (to save the trouble of maintenance and scaling costs). But I think software should be so flexible that it can live in each of those environments. Talk about utility computing: wouldn’t software have so much more utility if it worked everywhere instead of being beholden to whatever your software provider offers or what hardware you happen to have in front of you right now?

Fortunately this type of flexible software does exist. It is called Free and Open Source Softare (FOSS) and it is becoming ubiquitous. In fact, whether you know it or not, you are using FOSS software: Apache, the FOSS web server, runs this web site and indeed the majority of all web sites. WordPress, the blogging software we use here is also “everywhere” and you can purchase it from “cloud” utility providers or install, run, and modify it yourself. The list of important FOSS software goes on and on and this blog is dedicated to helping elucidate its importance as well as the issues involved in managing it.

So I would argue that instead of letting our heads go to the “clouds” we need to ask how can we make software that works in all environments, on all hardware, and for all people? … how can we make software that is comprehensively flexible?

Posted by CJ Fearnley in FOSS Community, 0 comments

Organizations Learning to Contribute to FOSS “The Right Way”

A couple weeks ago I wrote that I would be attending the 4th Annual Linux Foundation Collaboration Summit. I wrote about much of my experience there and at the Open Source Business Conference back in March over in my personal blog: “Lessons from Open Source Business Conference and the Linux Foundation Collaboration Summit”.

However, I also wanted to make a post here to cycle back to some of what I learned from the Collaboration Summit in relation to my March 30th post about contributing, “How and why contributing to FOSS can benefit your organization”. In this post I discussed using community tools, getting involved in the community and what steps you could take to get there. This was based upon several years of my own involvement in the FOSS (Free/Open Source Software) community directly and now my experience working for a company which makes FOSS contributions.

The talks at the Collaboration Summit strengthened my resolve in and increased the clarity of my understanding about the right way of going about contributing to FOSS as a company. At this conference there were multiple talks from major companies and figures within the FOSS business world which drove home the need for working with the community. All of these companies had stories about how they had tried to contribute to FOSS and struggled because they went about contributing as a walled off company rather than contributing just like other contributors did and using the same tools that contributors did.

A keynote which really stood out and succinctly discussed all of this was Dan Frye‘s talk, “10+ Years of Linux at IBM” (video). The first half of the keynote discusses the progress of Linux within IBM, but then he moves into discussing contributing itself. Some of their take-aways were that they needed to get involved directly with small contributions and do away with closed-door meetings and canned corporate responses, IBM employees were empowered to become community members. They needed to learn to collaborate with the community to develop higher quality solutions than they could have in-house, and to start these discussions with the community early in the brainstorming process. Related to collaboration, he also discusses control, and how a company does not have it within a community and needs to learn to deal with that, instead what a company should strive for is influence within a project to help guide direction and priorities. He also suggests never creating a project. Instead he encourages companies to join a project that’s close to what they need and work with them to take it in a direction that can benefit everyone and reach their goals and scratch their itches.

What struck me most at the conference regarding the subject of contributing is they are all reaching the same conclusions about the proper ways to successfully contribute. In the end, they learned that they must fully collaborate openly throughout development with the open source communities they’re working with.

Posted by Elizabeth Krumbach in Conference, FOSS Community, 0 comments

Attending the Linux Foundation Collaboration Summit 2010

On the heels of the 5th Annual Emerging Technologies for the Enterprise Conference (ETE 2010) in Philadelphia that CJ attended last week, I’ll be attending the 4th Annual Linux Foundation Collaboration Summit tomorrow through Friday in San Francisco.

The Linux Foundation Collaboration Summit is an exclusive, invitation-only summit gathering core kernel developers, distribution maintainers, ISVs, end users, system vendors and other community organizations for plenary sessions and workgroup meetings to meet face-to-face to tackle and solve the most pressing issues facing Linux today.

My attendance will be in my capacity as a member of the Ubuntu Community Council as well as my role as a Debian Systems Administrator. As such, my attention will be split at the summit between community and governance interests, like the FOSSBazaar Workgroup and Josh Berkus’ How to Prevent Community: Making Sure Your Pond Stays Small, and talks and panels like Does Open Source Mean Open Cloud? where Ubuntu founder Mark Shuttleworth will be a panelist, and the Linux Standard Base Workgroup and Virtualization discussions.

It’s shaping up to be an exciting summit, if you are also attending be sure to say “Hello”!

Posted by Elizabeth Krumbach in Conference, Debian, FOSS Community, News, Ubuntu, 0 comments