Multiple SSL Virtual Hosts on One IP With Server Name Indiction - HELP NEEDED

Discussion in 'Apache' started by seismicmike, Apr 5, 2010.

  1. #1
    Hey guys. I want to be able to run multiple name based https virtual hosts on the same IP address. I have several clients who all want their own ssl certificate and I can't afford to provide them with a dedicated server or IP.

    I'm using httpd 2.2.14
    My Distro is CentOS.

    The guide that initially got me started was this: http://wiki.apache.org/httpd/NameBasedSSLVHostsWithSNI

    From that guide I identified three things I needed to do.

    1. Upgrade OpenSSL. The version in the CentOS repos is 0.9.8e, and I need 0.9.8f or later with TLS compiled in (which k or later has by default).
    2. Either do 1 in such a way that the core openssl package is upgraded, or compile a custom build in /opt and rebuild httpd to use the custom build.
    3. Reconfigure apache in httpd.conf to use the SSLStrictSNIVHostCheck off option and configure the 443 hosts as name based vhosts.

    Initially I tried to patch the RPM from the CentOS repos and that didn't work, so I ended up compiling openssl on my own (0.9.8m) and putting it in /opt/openssl. Then I downloaded the source RPM of httpd and tried to reconfigure it.

    My plan was to change the option in ./configure where it sets --with-ssl to use the custom build of openssl in /opt. So I opened the spec file and found the following section:

    
    ../configure \
            --prefix=%{_sysconfdir}/httpd \
            --exec-prefix=%{_prefix} \
            --bindir=%{_bindir} \
            --sbindir=%{_sbindir} \
            --mandir=%{_mandir} \
            --libdir=%{_libdir} \
            --sysconfdir=%{_sysconfdir}/httpd/conf \
            --includedir=%{_includedir}/httpd \
            --libexecdir=%{_libdir}/httpd/modules \
            --datadir=%{contentdir} \
            --with-installbuilddir=%{_libdir}/httpd/build \
            --with-mpm=$mpm \
            --with-apr=%{_prefix} --with-apr-util=%{_prefix} \
            --enable-suexec --with-suexec \
            --with-suexec-caller=%{suexec_caller} \
            --with-suexec-docroot=%{contentdir} \
            --with-suexec-logfile=%{_localstatedir}/log/httpd/suexec.log \
            --with-suexec-bin=%{_sbindir}/suexec \
            --with-suexec-uidmin=500 --with-suexec-gidmin=100 \
            --enable-pie \
            --with-pcre \
            $*
    
    make %{?_smp_mflags}
    popd
    }
    
    # Build everything and the kitchen sink with the prefork build
    mpmbuild prefork \
            --enable-mods-shared=all \
            --enable-ssl --with-ssl --enable-distcache \
            --enable-proxy \
            --enable-cache --enable-mem-cache \
            --enable-file-cache --enable-disk-cache \
            --enable-ldap --enable-authnz-ldap \
            --enable-cgid \
            --enable-authn-anon --enable-authn-alias \
            --disable-imagemap
    
    Code (markup):
    It looks like the --with-ssl option is not part of the ./configure command, but part of this prefork command that I'm unfamiliar with. At any rate, I tried first to update that prefork command to be:

    
    mpmbuild prefork \
            --enable-mods-shared=all \
            --enable-ssl --with-ssl=/opt/openssl/ssl --enable-distcache \
            --enable-proxy \
    ...
    
    Code (markup):
    On later attempts I moved it up into the ./configure section.
    
    ../configure \
            --prefix=%{_sysconfdir}/httpd \
            --exec-prefix=%{_prefix} \
            --bindir=%{_bindir} \
            --with-ssl=/opt/openssl/ssl \
    ...
    mpmbuild prefork \
            --enable-mods-shared=all \
            --enable-ssl --enable-distcache \
            --enable-proxy \
    ...
    
    Code (markup):
    both methods successfully build me an RPM without errors, but once installed the SNI did not work as I hoped.

    After running rpmbuild -ba httpd.spec on the file, I got an RPM of httpd and of mod_ssl. I upgraded both of those using

    rpm -Uvh --replacepkgs --replacefiles httpd.rpm mod_ssl.rpm
    Code (markup):
    then I restarted HTTPD and no functionality had changed. At this point that's ok, because I still have to update the httpd.conf file.

    So I figure at this point, with the new version of openssl existing in /opt and httpd reconfigured to look for ssl there, I should be able to configure this bad boy to work properly. Right? Wrong....

    Here's the behavior I get. If I use a setup like the following:

    
    Listen 443
    
    NameVirtualHost *:443
    
    <VirtualHost *:443>
         ServerName site.com
         DocumentRoot "/path/to/htdocs"
         SSLEngine on
         SSLCertificateFile ssl.crt/site.com.crt
         SSLCertificateKeyFile ssl.key/site.com.key
    ...
    </VirtualHost>
    
    Code (markup):
    I get no errors whatsoever when I restart httpd, but I don't get the right SSL cert when I attempt to browse. I get some generic cert for "localhost.localdomain". I get this regardless of browser. The site does load, though, if I accept the bad cert.

    If I add the SSLStrictSNIVHostCheck off directive as the guide I linked instructed, I get the following error when restarting HTTPD:

    
    Starting httpd: Syntax error on line 982 of /etc/httpd/conf/httpd.conf:
    SSLStrictSNIVHostCheck failed; OpenSSL is not built with support for TLS extensions and SNI indication. Refer to the documentation, and build a compatible version of OpenSSL.
    
    Code (markup):
    I checked the guide and it said I might need to set an environment variable called LD_LIBRARY_PATH to the path of the custom SSL I built. So I tried using the export command from cli, editing /etc/httpd/conf/httpd.conf, /etc/httpd.conf.d/ssl.conf, /etc/sysconfig/httpd, and /etc/init.d/httpd to add that line (one at a time). Each time, it did not correct the error.

    I can think of three possible root causes:

    1. OpenSSL did not successfully build the TLS extension in when I did the custom build (thought that's supposed to be done by default)
    2. The ./configure that I tried to edit in the httpd.spec file did not work properly and HTTPD is still built to look for the core ssl program
    3. Even though the httpd package is built properly, the application is still looking for the old version of openssl at run time and I need to figure out how to set the LD_LIBRARY_PATH to the right path.

    Any suggestions on how to go about troulbeshooting and fixing this would be very very welcome. Thank you in advance.

    Specific Questions:

    1. Is there a way to query a built RPM package to find out what options were used on the ./configure line?
    2. Where/how do I set that LD_LIBRARY_PATH variable for run time?
     
    Last edited: Apr 5, 2010
    seismicmike, Apr 5, 2010 IP
  2. seismicmike

    seismicmike Peon

    Messages:
    63
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #2
    I recompiled openssl using version 1.0, but I'm still having the same behavior....
     
    seismicmike, Apr 5, 2010 IP
  3. seismicmike

    seismicmike Peon

    Messages:
    63
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #3
    I have new info... when I run

    
    ldd /usr/sbin/httpd
    
    Code (markup):
    the output does not include libssl at all....
     
    seismicmike, Apr 5, 2010 IP
  4. seismicmike

    seismicmike Peon

    Messages:
    63
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #4
    shameless self bumping
     
    seismicmike, Apr 6, 2010 IP
  5. seismicmike

    seismicmike Peon

    Messages:
    63
    Likes Received:
    0
    Best Answers:
    0
    Trophy Points:
    0
    #5
    Well I guess what I've really been looking for is the envvars file. On my personal Ubuntu machine there's a file /etc/apache2/envvars, but no such file exists on the CentOS server I'm trying to configure, so I did a locate envvars and found that there are a couple files with that name in the BUILD folder of my rpmbuild environment, namely the httpd-2.2.14/support/envvars-std.in file that generates a couple other files, for example httpd-2.2.14/event/support/envvars-std. So I guess the settings of envvars are compiled into the build of the CentOS server.

    So I checked the generated envvars-std files and they have this:

    
    LD_LIBRARY_PATH="/usr/lib64:$LD_LIBRARY_PATH"
    export LD_LIBRARY_PATH
    
    Code (markup):
    As stated above, I need this LD_LIBRARY_PATH to have the location of my custom SSL build /opt/openssl/lib64... My question is, do A) I want to add it onto this, so to have this:

    
    LD_LIBRARY_PATH="/opt/openssl/lib64:/usr/lib64:$LD_LIBRARY_PATH"
    export LD_LIBRARY_PATH
    
    Code (markup):
    Or B) do I want to replace the /usr/lib64 to have:

    
    LD_LIBRARY_PATH="/opt/openssl/lib64:$LD_LIBRARY_PATH"
    export LD_LIBRARY_PATH
    
    Code (markup):
    Here's what I'm thinking about those two scenarios:

    In A, is it possible that that will cause HTTPD to find both versions of OpenSSL and make it confused as to which to use?
    In B, is it possible that there are other things besides OpenSSL in that /usr/lib64 directory that are going to be missing now?

    So which do I do? Does the order that I put them in matter? Like so if I put A like I have it written above, will it check /opt/openssl/lib64 first and if it doesn't find what it needs, go on to /usr/lib64, or if I put them the other way, it'll go to /usr before /opt?

    Or should I not use /opt/openssl/lib64, and instead use another directory in there? (bin,ssl?)

    Thanks for any help you can provide.

    As an aside: another thing I've discovered that I was doing wrong is that I needed to compile OpenSSL with the "shared" flag in ./config so that it creates the necessary .so files. Those had been missing before.

    Thanks again.
     
    seismicmike, Apr 7, 2010 IP