Solaris on DELL PowerEdge Servers

[Ken Mays]

These Dell PowerEdge Servers are SOLARIS CERTIFIED as reported in the HCL by Oracle for Dell as of April 2010. The Dell website is a bit outdated on the Dell PowerEdge certification status with Oracle Solaris 10u8.

Ref:
http://www.sun.com/bigadmin/hcl/search.jsp
http://advisors.dell.com/advisorweb/iDriveMatrixView.aspx?redirect=1

"Dell Supported: Dell Engineering has successfully tested and validated only the current operating system (OS) version on Dell PowerEdge Servers. Dell adheres to all published partner test plans to ensure OS compatibility and provides full technical support for the Dell hardware components running on the OS. Dell Supported systems do not have OpenManage support."

Dell certified and supported servers with Solaris 10u8:

Dell M605,M610,M710,M805,M905,M910

Dell R210,R300,R310,R410,R510,R610,R710,R805,R810,R900,R905,R910

Dell T110, T310, T410,T610,T710

Note: I've reviewed and tested Oracle OpenSolaris 2009.06 on Dell PowerEdge T110, T310, T410,T610, T710 and R900 servers with Oracle HCTS 5.0. No major problems reported yet outside of the provided HCL info. Any HCTS 5.x certifications on OpenSolaris binary release distributions (i.e. OpenSolaris 2010.05) will probably come in the near future from Dell.

I'll let Oracle or Dell correct me otherwise.

~ Ken Mays

[Chavdar Ivanov]

It took me less than half an hour a few days ago to configure an R610 with snv_134 - no problems whatsoever. I even kept the hardware RAID1 on (it was possible the system to be reused for something else, normally I would just JBOD it and configure the usual ZFS mirror).

Chavdar Ivanov

[James Nelson]

hmm no problems? We have svn_124 running on a R610. Install was fine, no special tricks etc.. and we have perc card in there. However there is a network bug with the bnx driver that will cause the network to hang under load. It happens randomly, sometimes once a week, sometimes once every few months... If you look up the bug reports others running 134 are having same issue.

[Chavdar Ivanov]

Post by James Nelson
hmm no problems? We have svn_124 running on a R610.
Install was fine, no special tricks etc.. and we
have perc card in there. However there is a network
bug with the bnx driver that will cause the network
to hang under load.

I ran it for a week or so as a Comstar iSCSI target and initiator; it should have loaded the network adapter enough...

Post by James Nelson
It happens randomly, sometimes
once a week, sometimes once every few months... If
you look up the bug reports others running 134 are
having same issue.

Chavdar

[James Nelson]

Just out of curiosity what firmware version do you have on the network card from dell? It looks like people with the 4.x version are not experiencing the errors, but 5.x people are.

do dmesg|grep bnx and look for the following line:
Jun 22 08:23:43 ipaapp bnx: [ID 995108 kern.info] NOTICE: bnx1: BCM5709 device wit h F/W Ver5000006 is initialized.

[Bruno Sousa]

Post by James Nelson
Just out of curiosity what firmware version do you have on the network card from dell? It looks like people with the 4.x version are not experiencing the errors, but 5.x people are.
Jun 22 08:23:43 ipaapp bnx: [ID 995108 kern.info] NOTICE: bnx1: BCM5709 device wit h F/W Ver5000006 is initialized.

I can say i had that issue..so now i run r410 and r710 with broadcom fw
4.x . However i know there's a bug fix in builds beyond 134 version.
So for now..just downgrade the FW of the broadcom.

Bruno

[Bruno Sousa]

Hi,

I did the downgrade using the setup utility where you usually upgrade
the drivers..however there's also the possibility of downgrade , and
that's what i have done.
Here's the output of a booting server :

bnx0: BCM5709 device with F/W Ver4060004 is initialized

Bruno

So you are not seeing the issue on version 4 of the firmware? Which version
of 4 do you have? How did you downgrade the firmware, did you use the dell
utility cd? I have read other threads that say it is a c-state issue and
that firmware doesn't matter? Don't know what to think, this is a
production server that I don't want to muck with too much.
-----Original Message-----
Sent: Wednesday, July 14, 2010 11:19 AM
To: James Nelson
Subject: Re: [osol-discuss] Solaris on DELL PowerEdge Servers

Post by James Nelson
Just out of curiosity what firmware version do you have on the network

card from dell? It looks like people with the 4.x version are not
experiencing the errors, but 5.x people are.

Post by James Nelson
Jun 22 08:23:43 ipaapp bnx: [ID 995108 kern.info] NOTICE: bnx1: BCM5709

device wit h F/W Ver5000006 is initialized.
I can say i had that issue..so now i run r410 and r710 with broadcom fw
4.x . However i know there's a bug fix in builds beyond 134 version.
So for now..just downgrade the FW of the broadcom.
Bruno

[David Anderson]

Disabling C-States in the bios will fix the broadcom network driver issue
under 5.XX firmware also. If you google enough someone has a patched driver
for snv_134 but I didnt really want to put it in my system so I just went
with disabling C-States in the bios.


Dell R610 running OSOL SNV134 with iscsi/nfs load and no network troubles
for about 14 or 15 days now.

Sorry if you got this twice, i seem to have missed the list with my first
reply.

Post by Bruno Sousa
Hi,
I did the downgrade using the setup utility where you usually upgrade
the drivers..however there's also the possibility of downgrade , and
that's what i have done.
bnx0: BCM5709 device with F/W Ver4060004 is initialized
Bruno

So you are not seeing the issue on version 4 of the firmware? Which

version

of 4 do you have? How did you downgrade the firmware, did you use the

dell

utility cd? I have read other threads that say it is a c-state issue and
that firmware doesn't matter? Don't know what to think, this is a
production server that I don't want to muck with too much.
-----Original Message-----
Sent: Wednesday, July 14, 2010 11:19 AM
To: James Nelson
Subject: Re: [osol-discuss] Solaris on DELL PowerEdge Servers

Post by James Nelson
Just out of curiosity what firmware version do you have on the network

card from dell? It looks like people with the 4.x version are not
experiencing the errors, but 5.x people are.

Post by James Nelson
Jun 22 08:23:43 ipaapp bnx: [ID 995108 kern.info] NOTICE: bnx1: BCM5709

device wit h F/W Ver5000006 is

initialized.

I can say i had that issue..so now i run r410 and r710 with broadcom fw
4.x . However i know there's a bug fix in builds beyond 134 version.
So for now..just downgrade the FW of the broadcom.
Bruno

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
_______________________________________________
opensolaris-discuss mailing list

[Chavdar Ivanov]

Post by James Nelson
Just out of curiosity what firmware version do you
have on the network card from dell? It looks like
people with the 4.x version are not experiencing the
errors, but 5.x people are.
Jun 22 08:23:43 ipaapp bnx: [ID 995108 kern.info]
NOTICE: bnx1: BCM5709 device wit
h F/W Ver5000006 is
initialized.

Ver500000b on mine, six of them, no problem at this time (I thought that the machine has been used for something else; it turned out it has been running the same network tests - initiator against Nexenta target and also target itself to some Hyper-V servers - all that time). Only two of the six were in use, though.


Chavdar

[Edward Ned Harvey]

Post by Ken Mays
These Dell PowerEdge Servers are SOLARIS CERTIFIED as reported in the

For what it's worth, this has been my experience with sol10/osol on dell
servers:

Getting sol 10 installed sometimes requires special tricks, because the
driver for the perc raid controller isn't included, or the sata optical
drive. Or both. So far, I have always been able to enable the sata optical
drive by entering bios and changing the sata mode to ATA or compatible or
whatever. Before I learned that, I followed internet blogs that said
install from an external USB optical drive. So far, I have always been able
to download the perc driver ISO from support.dell.com and then the
installation process is like this ... You boot from the sol10 CD. At some
point you hit "5" or "F5" to load special drivers from another ISO. And
then things progress fine.

OpenManage is not available on solaris/opensolaris, so you're lacking the
software utility to reconfigure the perc card while the OS is running. This
is particularly important if you wish to replace a failed disk without
shutting down into BIOS to configure the perc. Instead, MegaCli is
available. It's very hard to find, and very difficult to use.

Getting osol installed only requires one special trick: Change the optical
to ATA mode, since the CD doesn't have the SATA driver for that optical
drive. Afterward, the same comments apply, regarding perc, openmanage, and
MegaCli.

I definitely endorse the use of sol10/osol on Dell hardware, if you need to
save money compared to sun/oracle, and you don't need the more powerful
hardware (beyond what dell offers). It's not as great as using Sun/Oracle
hardware, but if the wimpier dell servers are powerful enough for you ...
then it's vastly lower cost.

[Mark van Kerkwyk]

I notice that the R510 isn't actually certified.

We have installed it without any modifications at all and there are no missign drivers at all.

I wonder why the R510 seems to miss out on the certifications when the R310 & te R610 are certified ??

[Edward Ned Harvey]

Post by Mark van Kerkwyk
I notice that the R510 isn't actually certified.
We have installed it without any modifications at all and there are no
missign drivers at all.
I wonder why the R510 seems to miss out on the certifications when the
R310 & te R610 are certified ??

I wonder ...

When I installed on a 2970 or R710 (I forget which or both) ... I had two
problems to overcome. First was the perc driver. I had to download a
special CD from support.dell.com, and hit something like the "5" key during
installation, to load additional drivers from a second CD ... And the 2nd
problem was the SATA CD drive. I had to either (a) put the CD drive into
ATA or Legacy or Compatible mode, or (b) use an external optical drive. I
actually did both of these; I just forget which soutions for which systems,
and never tested extensively to see if both solutions worked on both
systems... I did just enough to get past the problems, then moved on and
forgot about it.

I wonder how universal or consistent the PERC and/or SATA problem is, to
installing solaris on Dell servers. For that matter, I wonder how universal
the solution is. ;-)

Knowing only how much I know right now today, I think the universal solution
that I would recommend and expect it to work optimally every time... is to
download a CD for the perc driver, and put the optical drive into ATA mode.