I have been teaching OS internals courses for many years, starting with Bell Labs/AT&T Unix System III in 1982, onto System V, SVR2, SVR3, SVR4, and Solaris Unixes since 1994. Along the way, I have also taught HP-UX internals, various device driver courses, and kernel debugging courses. I started using unix on the Sixth Edition in 1975. I have also done a fair amount of kernel development and debugging, along with some user level stuff.
The audiences I have for internals courses has been quite varied. Many of the people I have taught have been in support or sustaining organizations, but I have also taught developers, system administrators, Java programmers, QA people, hardware engineers, and even end users. Along the way, I have been asked by various people (many of them managers), "Why should I or my team take this course? What will I or my team get out of this training?"
In response to the first question, I usually tell people that an internals course should teach students how the system works, and why it works the way it does. In other words, the course teaches the data structures and algorithms used by the operating system to manage the resources of the computer, and explains the architecture of the system, as well as the rationale behind the design decisions that have been made to implement the system. My view is that knowledge of how the system works can benefit everyone. For developers (especially kernel developers), knowledge of the system is key to adding new functionality. For system administrators, knowledge of the OS can help to do troubleshooting and performance analysis. Tools like DTrace become even more useful when one has knowledge of what's going on in the system. In general, knowledge of how the system works allows everyone who uses the system to make better use of the system.
As for what specific skills are acquired in an internals course, I make very extensive use of tools that come with the system during the training. Both when I am lecturing, and in lab work. My view has always been that in order to learn the concepts being taught, one must be able to actually "see" them. Tools like DTrace, mdb, kmdb, and other observability mechanisms are key to doing this. I do not "teach" the tools, but rather we use the tools in lots of examples throughout the course. At the end of the course, I am satisfied if my students can start to learn things on their own. Basically, a good internals course should be an "enabling" course. It should enable the student to learn more on their own. For some, they may never use the specific tools used during the class in the specific ways they are used, but it will educate students that one can actually determine what the system is doing at any given time. For others, they will be using the tools consistently in their work.
As with all training, you only get out of it what you put into it.
If you're interested in Internals training, please visit training from joyent.
I hope to see people in class soon!