Training Example: illumos – Review the Data, Give Your Score & Compare to the Real AI Evaluation

[H2] What is illumos?

illumos is a Unix operating system which provides next-generation
features for downstream distributions, including
advanced system debugging,
next generation filesystem, networking, and virtualization options.

illumos is developed by both volunteers and companies building products
on top of the software.
illumos is an excellent base for both traditional and cloud-native
deployments.

[H2] Getting Started

[H3] Download and install
The OmniOS,
OpenIndiana,
and
Tribblix
distributions are a good place for new users to get started.
You can install in a virtual machine or on bare metal.
Look at the
full list to find a distribution that meets your needs!

[H3] Get the source
illumos is freely available from our
source repository, or a read-only
mirror on GitHub. You can see recent changes
here.
Instructions for building illumos on various distributions
can be found
here!

[H3] Join the community

Discussions occur on the project mailing lists;
you can join and post, or read the archives on the web.
The #illumos channel on Libera Chat
(IRC) is very active and has online logs.
We strive to be inclusive, and enforce a
code of conduct in our interactions.

[H2] Other Resources

There are many online resources that describe the illumos project. Here
are some important links:

Code
Of Conduct
Contribution Guide
Manual Pages
Online Books
Bug Tracker
Mailing Lists
illumos
Source on Gerrit (primary repository)
illumos Source on
GitHub (read-only mirror)
illumos OpenGrok Source Browser
illumos Project Discussions
(IPD)
Documentation
Gerrit (Code Reviews)
Supported Hardware Database
Fault Management Message Database

[H1] Contributing to illumos¶
[H2] Overview¶
This document describes our contribution process in some detail. If you need
guidance or assistance, especially if you are new to the project, you can
always reach out to the developer mailing list or the
IRC channel for help.
Contributing to a large project with a long history like illumos can seem
daunting! To get started, you might consider watching Ryan
Zezeski going through fixing a bug in ZFS in this
"illumos Day" talk, which includes submitting a patch for integration into
illumos-gate.git:
video
slides
RTI Process
The Request To Integrate (RTI) process is also documented in the illumos
Developer's Guide.
[H2] Finding An Area To Contribute To¶
If you're not sure what you want to work on, you can start by looking at our
list of bite-size
bugs which
should be easy for newcomers to pick up. You can also look at the entire list
of issues and see if any with status "New" fit your skill set.
If you are adding a new feature or addressing a problem not currently on our
list of issues, please
create a new issue
describing it. You need to create an account in the bug tracker in order to be
able to file issues.
Before you start working on the code, it is advisable to first ask on either
the developer mailing list or the IRC
channel for advice on the particular area you're
interested in. For larger bodies of work, it is worth writing an illumos
Project Discussion (IPD) document that
describes your plan and seek feedback from core team members and others.
[H2] Writing The Code¶
Our guide to building illumos covers setting up a
build environment and getting the source. To make sure you have a functional
build environment, it helps to try at least one build without modifying
anything. This also helps you to familiarize yourself with the build system
and source tree layout.
An unparalleled resource for understanding the operating system is the book,
Solaris Internals: Solaris 10 and OpenSolaris Kernel
Architecture.
We have an OpenGrok source browser and search engine
which is very useful for development.
Feel free to ask the developer mailing list or the
IRC channel for help.
We encourage you to "commit early, commit often" as you work, using your
personal clone of illumos-gate.git.
[H2] Testing¶
The illumos gate is a core technology leveraged by several commercial vendors
that rely on it to be a stable, well-tested platform. As such, changes to
illumos require qualified code review from community peers. A reasonable
effort must also be made to test each change, to ensure that the change is
stable enough for others to rely upon.
The core team member looking at your
integration request will discuss testing with you, and will ultimately decide
when your testing is sufficient.
The scope of required testing depends on the nature of the change and a
discussion with the core team member looking at your change, but in general the
core team adopts a "shrink to fit" mentality. Below are some examples of a
reasonable level of testing based on the collective experience of the current
core team. A core team member may ask for more testing depending on the
nature of the change.
[H3] All changes¶
Make sure you test the results of the illumos-gate build with only your
changesets applied. To do this, you will need to build the illumos
code. It is generally insufficient to test the build
results of a downstream fork that contains other non-trivial changes as there
may be implicit dependencies that break other consumers of the core illumos
gate.
If your code has been running in production as part of a downstream
distribution of illumos, or has already gone through other testing, please
include that information. The core team will determine if any additional
testing is required for integration.
[H3] Changing a single program¶
If your change affects only a single binary (e.g., /usr/bin/ls), it is likely
sufficient to copy the binary to a system running a distribution with a
reasonable similarity to vanilla illumos-gate and running your tests there.
These tests should cover enough functionality to demonstrate the correctness of
your changes, and could involve stress tests in addition to functional tests
depending on the nature of the change. This assumes that the binary has a
trivial set of library dependencies that are reasonably standard across
distros.
[H3] Changing the implementation of a library¶
If your change affects the implementation of a library, but does not change any
externally visible interfaces, you should run tests against a reasonable subset
of programs that exercise the code in question. As with a single binary, it is
likely sufficient to copy this library to a system running a distribution which
has identical interfaces to the vanilla illumos-gate.
Some core libraries (most notably libc) are challenging to replace on a live
system, so you will likely need to bootstrap a complete new image or boot
environment of your distribution in order to be able to test your changes. How
to do this depends on your distribution; e.g., OpenIndiana or OmniOS can use
onu to update to new packages from a nightly build.
[H3] Changing the implementation of a driver or adding a new driver¶
If your change affects a single driver, you may or may not be able to get away
with copying that driver to a system running an alternate distribution. On one
hand, many of the kernel interfaces (e.g., the DDI) are likely identical, but
if the core implementation of that distribution has diverged significantly from
illumos-gate, you may be asked to bootstrap a complete environment from your
vanilla illumos build. Driver tests should demonstrate a reasonable amount of
functional and stress testing on appropriate hardware.
[H3] Changing the core kernel or multiple interconnected binaries¶
If you are making a change to the core kernel (e.g., unix or genunix), or
have a series of changes across a number of libraries, commands, or kernel
components, they need to be tested as a cohesive whole. You must bootstrap a
usable environment from your illumos-gate build that is sufficient to
demonstrate the correctness of your changes.
Exactly what "bootstrapping" means is dependent on the distribution, but the
goal is to run the entirety of the your illumos-gate build (not a combination
of downstream and upstream illumos-gate bits) in a testable environment. Not
all distros are capable of bootstrapping a vanilla illumos-gate build; check the
distribution list and ask on the developer
list if you are unsure of how to do this.
For example, on OpenIndiana or OmniOS, you can use onu to install the
packages produced by a nightly build. On a SmartOS system, you can build
a wholly new "platform image" and boot a system using it.
The scope of testing here will likely be more substantial, and should focus on
stability of the system as well as the functionality of the changed
component(s).
As always, contact the core team member (or the core team mailing list if you
don't yet have someone in mind) if you have more questions about what
constitutes sufficient testing for a particular change. You are welcome to ask
for help in creating a test plan well in advance of completing the work on you
change, so that the core team can help set expectations and provide feedback
throughout the process.
[H2] Code Review¶
At least one person (other than you, of course) should review your changes.
These reviewers should have demonstrated expertise in the area in question or
are trusted as knowledgeable by the illumos community at large. If you work for
a commercial entity, it is best to include reviewers from outside your company,
but this is not required.
The core team member looking at your
integration request will discuss the review you have received with you, and
will ultimately decide whether your change has undergone sufficient review.
If you don't know how to find adequate review, you may post the changes to the
developer mailing list to ask for feedback, or ask the core team if there is
anyone in particular they'd like to see review the code.
When seeking review, it is generally best to upload to some web-based review or
source hosting system, rather than attach a patch to your list e-mail. The
illumos project provides a Gerrit instance which
you can use with your illumos bug tracker account credentials. For more
information about using Gerrit, see our documentation on Gerrit.
Remember to mention the issue ID in the email -- if one does not yet exist,
please create one!
[H3] Ensuring adequate review¶
It is up to you as a contributor to gather reasonable review to satisfy a
member of the core team that the code is correct and of high quality. Exactly
what this means will vary based on the nature and scope of the change, but here
are some guidelines that will give the core team confidence in your changes.
There should be at least one code reviewer (focused on the implementation
details) that is different from the core team member (focused on ensuring
overall quality of contributions).
For non-trivial changes to major technology areas (ZFS, DTrace, etc), at
least one of the reviewers should be regarded as a subject matter expert by
the community.
Changes with significant impact should participate in a public review when
appropriate, or otherwise ensure that alternate opinions are represented.
When there are disagreements, every attempt should be made to adequately
address reviewer's comments. In the event they cannot, it is acceptable to
submit an RTI with known dissent. Such disagreements must be noted in the
RTI submission with an explanation as to why the review comments were not
addressed.
When in doubt, more review is generally better. If anything is unclear, or
you're stuck on something, you can always ask the core team for guidance.
[H2] Submitting A Patch¶
When you have adequate code review and test results for your proposed change,
you prepare a "request to integrate" (RTI).
First, in the issue(s) you created in the bug tracker, make sure you have:
Any analysis that lead to the fix; e.g., if you used DTrace or MDB to
determine what went wrong it'd be good to include scripts and/or output
from your debugging session
Your testing plan and results from your testing. This could be simple
for simple changes; e.g., "I ran the updated ls binary and this is
the new output". It could also be more detailed for a complicated bug.
Providing the testing notes in the issue allows the core team to easily
see what you did to test, and allows future engineers to better understand
how to test their changes in the same area of the system.
You should have a full clean build of the gate with your changes applied. This
currently means a successful build with GCC 10 as the primary compiler, and
GCC 14 and smatch as shadow compilers, with no warnings or errors in the
resulting mail_msg. Note that mail_msg includes a diff against the previous
build, so your patch's clean build should come directly after a clean build of
the previous commit.
If your commit does not yet include "Reviewed by:" indicating your reviewers,
this is when you should add those lines. We do not have a blank line between the
first line of a commit message in illumos, and the following "Reviewed by:"
lines. If you run git format-patch to get a patch to include, this will
result in an oddly-formatted "Subject:" line - that is expected and OK.
!!! note Assembling "Reviewed by:" lines
If your review is not through Gerrit, you should have contact information
for your reviewers that can be included when adding "Reviewed by:" lines. If
your review is through Gerrit, you can use the Gerrit /changes
API
to get reviewer information and reduce the risk of a transcription error for
your reviewers' names or e-mail addresses. This jq is an example of how you
might want to assemble "Reviewed by:" lines:
```
curl -s "https://code.illumos.org/changes/$YOUR_CHANGE_NUMBER?o=LABELS&o=DETAILED_ACCOUNTS" | \
tail -n 1 | \
jq -r '.labels."Code-Review".all[] |
select(.value==1) |
"Reviewed by: \(.name) <\(.email)>"'
```
If your change has been reviewed on Gerrit, you (or git pbchk) may have found
trivial changes such as whitespace nits, comment spelling, or copyright dates
since the last round of review. Please ensure any changes at this point are
also pushed to Gerrit. If your changes on Gerrit are the same as you would
include as a patch for integration, you can omit the patch from your RTI e-mail
entirely; the core team can fetch your change from Gerrit as well.
Your RTI e-mail should include:
The link to the illumos issue(s) you're fixing, e.g.,
https://illumos.org/issues/10052
The changes that were reviewed; e.g., a link to your Gerrit
review, or an attached patch otherwise.
The full "change set description" (i.e., git whatchanged -v origin/master..)
including:
Issue number(s) and description(s)
Reviewed by: First Last <first.last@example.com> lines
List of files affected
Output of git pbchk (run under bldenv or have /opt/onbld/bin in PATH).
This is optional if git pbchk prints nothing and exits with 0
An attached clean mail_msg from a full nightly build (including shadow
compilers, as noted above)
Information about how the changes were tested (it's sufficient to
mention that the testing notes appear in the bug tracker)
Here is an example change description:
123 Description of the issue in our tracker
Reviewed by: Jack <jack@eng.sun.com>
Reviewed by: Ohana Matsumae <ohana@kissui.ishikawa.jp>
Note this description does not include a "Change-ID:" line - it is the
description the commit should have when integrated, minus "Approved by:". Since
Gerrit identifies changes with the "Change-ID:" line, this will be a little
different from the description in the associated Gerrit link. This is okay! If
your RTI is approved, the core team member integrating your patch will remove
this line from the description when they add an "Approved by:" recording their
approval.
Note
You can use git commit --amend to edit the commit message.
A core team member will need to judge whether
your code review and testing are adequate for the scope of changes you propose.
Note that the core team members's job as part of integration is not necessarily
to review your code again, only to judge whether review and testing was
adequate.
When you are ready, send an email to the illumos core team (formerly
"advocates") mailing list.
Though you are welcome to subscribe to the list, it is not required; you can
just send mail to advocates@lists.illumos.org. If you're interested in what
past RTIs looked like, you can browse past
messages.
A reasonably complete example issue is 10052 "dladm
show-ether" should pick one kstat snapshot and stick with
it:
Bug #10052 in the bug tracker includes both analysis and testing notes
The RTI mailing list thread shows the sort of interaction you can expect
Note
Your email needs to be less than 2MB in size. If your change is
large, compress the patch or make it

[H1] illumos: manual sections

[H2]
section 1: User Commands
intro.1
Section Introduction

man1

User Commands

man1b

BSD Compatibility Package Commands

man1c

Communication Commands

man1has

User Commands

man1onbld

illumos Build Tools

man1s

illumos Specific Commands

[H2]
section 2: System Calls
intro.2
Section Introduction

man2

System Calls

[H2]
section 3: Libraries
intro.3
Section Introduction

man3

Introduction to Library Functions

man3avl

AVL Tree Library Functions

man3bsm

Security and Auditing Library Functions

man3c

Standard C Library Functions

man3cfgadm

Configuration Administration Library Functions

man3commputil

Communication Protocol Parser Utilities Library Functions

man3contract

Contract Management Library Functions

man3cpc

CPU Performance Counters Library Functions

man3curses

Curses Library Functions

man3dat

Direct Access Transport Library Functions

man3devid

Device ID Library Functions

man3devinfo

Device Information Library Functions

man3dlpi

Data Link Provider Interface Library Functions

man3elf

ELF Library Functions

man3exacct

Extended Accounting File Access Library Functions

man3ext

Extended Library Functions

man3fcoe

FCoE Port Management Library Functions

man3fstyp

File System Type Identification Library Functions

man3gen

String Pattern-Matching Library Functions

man3gss

Generic Security Services API Library Functions

man3head

Headers

man3iscsit

iSCSI Management Library Functions

man3jedec

man3kstat

Kernel Statistics Library Functions

man3kvm

Kernel VM Library Functions

man3ldap

LDAP Library Functions

man3lgrp

Locality Group Library Functions

man3lib

Interface Libraries

man3m

Mathematical Library Functions

man3mail

User Mailbox Library Functions

man3malloc

Memory Allocation Library Functions

man3mp

Multiple Precision Library Functions

man3mpapi

Common Multipath Management Library Functions

man3mvec

man3nsl

Networking Services Library Functions

man3nvpair

Name-value Pair Library Functions

man3ofmt

Formatted Output Functions

man3pam

PAM Library Functions

man3papi

PAPI Library Functions

man3perl

Perl Library Functions

man3picl

PICL Library Functions

man3picltree

PICL Plug-In Library Functions

man3pool

Pool Configuration Manipulation Library Functions

man3proc

Process Control Library Functions

man3project

Project Database Access Library Functions

man3resolv

Resolver Library Functions

man3rpc

RPC Library Functions

man3rsm

Remote Shared Memory Library Functions

man3sasl

Simple Authentication Security Layer Library Functions

man3scf

Service Configuration Facility Library Functions

man3sec

File Access Control Library Functions

man3secdb

Security Attributes Database Library Functions

man3sip

Session Initiation Protocol Library Functions

man3slp

Service Location Protocol Library Functions

man3socket

Sockets Library Functions

man3stmf

SCSI Target Mode Framework Library Functions

man3sysevent

System Event Library Functions

man3tecla

Interactive Command-line Input Library Functions

man3tsol

Trusted Extensions Library Functions

man3utempter

UTEMPTER Library Functions

man3uuid

Universally Unique Identifier Library Functions

man3volmgt

Volume Management Library Functions

man3xcurses

X/Open Curses Library Functions

man3xnet

X/Open Networking Services Library Functions

[H2]
section 4: Device and Network Interfaces
intro.4
Section Introduction

man4

Device and Network Interfaces

man4d

Devices

man4fs

File Systems

man4i

Ioctl Requests

man4ipp

IP Quality of Service Modules

man4m

STREAMS Modules

man4p

Protocols

[H2]
section 5: File Formats and Configurations
intro.5
Section Introduction

man5

File Formats and Configurations

[H2]
section 7: Standards, Environments, and Macros
intro.7
Section Introduction

man7

Standards, Environments, and Macros

[H2]
section 8: Maintenance Commands and Procedures
intro.8
Section Introduction

man8

Maintenance Commands and Procedures

[H2]
section 9: Kernel Concepts
intro.9
Section Introduction

man9

Kernel Concepts

man9e

Driver Entry Points

man9f

Kernel Functions for Drivers

man9p

Kernel Properties for Drivers

man9s

Data Structures for Drivers