No support for MongoDB 6.0 on Ubuntu 22.04 (jammy)

[rogermartensson]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.28
• Ruby: ?
• Distribution: Ubuntu 22.04 jammy
• Module version: 4.2.0

How to reproduce (e.g Puppet code you use)

Try to install MongoDB 6.0

What are you seeing

Nothing is installed. Wrong APT-source with no key installed.
(Maybe need too look into new way to install keys)

What behaviour did you expect instead

Support for MongoDB 6.0

[witjoh]

@rogermartensson #677
If possible can you give a hand to verify this PR on Ubuntu ?
(I only tested it on redhat based distro's)

[rogermartensson]

Late-Reply-Reason: Vacation

I will take a look. My guess there will be some differences. do mind that I am not an mongodb or puppet expert.

[rogermartensson]

I've started to look into this and to begin with I started to look into using Ubuntu 20.04 and MongoDB 4.4 which is what we are running in production right now.

This to look into if I need to change any of our own Puppet code in this deployment.

I am having the same type problems setting up at replica set as reported in your PR.
Somehow it will not initialize the replicatset even if it says it does.

Notice: /Stage[main]/Profile::Mongodb::Node/Mongodb_replset[rs1]/ensure: created

and it will error out with:

Error: /Stage[main]/Profile::Mongodb::Node/Mongodb_replset[rs1]: Could not evaluate: Can't connect to any member of replicaset rs1.

Before this message and before the 'created' message I get this message three times.

Debug: Request failed: 'Execution of '/usr/bin/mongosh admin --quiet --host mongodbnode1.example.org --eval "load('/root/.mongoshrc.js'); EJSON.stringify(rs.status())"' returned 1: MongoServerError: no replset config has been received' Retry: '3'

  class { 'mongodb::globals':
    version             => '4.4.15'
    manage_package_repo => true,
    use_enterprise_repo => false,
  } -> class { 'mongodb::client':
  }

    class { 'mongodb::server':
      dbpath        => $db_path,
      bind_ip       => ['0.0.0.0'],
      port          => 27017,
      verbose       => true,
      replset       => 'rs1',
      # set_parameter => $c,  # set_parameter used to insert enableLocalhostAuthBypass: false
      require       => Class['mongodb::globals'],
    }

# To create the replica set I use
        mongodb_replset { 'rs1':
          ensure  => present,
          members => $nodenames,
        }

I have tried using MongoDB 6 and it seems to install it correctly but it has the same problem as described above so I didn't test it very deeply.

[rogermartensson]

Did some more testing and it looks like it may work if I, after a new install, manually run rs.initiate() on the first node.
More testing needed.

[witjoh]

There need some more work on this PR indeed. Hadn't that much time recently, but picking this up again ...
Thanks for the feedback and testing.

Installing a new cluster right now on redhat. Will pay close attention about the rs.initate() ...

[rogermartensson]

When initializing I have set:

enableLocalhostAuthBypass: false

This is removed after replset and first admin user is created.

I did get some other errors but my take on those is that I needed to use host:port when adding members with mongo_replset.

I have installed mongodb on all nodes in the replica set before initializing.
The command I use to initialize is.:

rs.initiate( {
   _id : "rs1",
   members: [
      { _id: 0, host: "n1.example.test:27017" },
      { _id: 1, host: "n2.example.test:27017" },
      { _id: 2, host: "n3.example.test:27017" }
   ]
})

After that I can use mongo_replset, mongo_user etc.

BTW, this is with MongoDB 4.4. I will try it with 6.0 soon. Ubuntu 22.04 (jammy) has support from version 6.0.4 if you need to add any extra code for that.

[flepoutre]

Hello,
Same issue with MongoDB 7x, isn't it ?

Error: Could not prefetch mongodb_database provider 'mongodb': 783: unexpected token at 'DeprecationWarning: .) is deprecated. Use .setReadPref("primaryPreferred" instead Setting read preference from "primary" to "primaryPreferred" { databases: [ { name: 'admin', sizeOnDisk: '40960', empty: false }, { name: 'config', sizeOnDisk: '12288', empty: false }, { name: 'local', sizeOnDisk: '73728', empty: false } ], totalSize: '126976', totalSizeMb: '0', ok: 1 } ' Warning: /Stage[main]/Mongodb::Server/Mongodb::Db[admin]/Mongodb_database[admin]: Skipping because provider prefetch failed Notice: /Stage[main]/Mongodb::Server/Mongodb::Db[admin]/Mongodb_user[User admin on db admin]: Dependency Mongodb_database[admin] has failures: true Warning: /Stage[main]/Mongodb::Server/Mongodb::Db[admin]/Mongodb_user[User admin on db admin]: Skipping because of failed dependencies Warning: /Stage[main]/S_mongodb::Zabbix/S_mongodb::Users[zabbix]/Mongodb_user[Create user zabbix on database admin]: Skipping because of failed dependencies Info: Stage[main]: Unscheduling all events on Stage[main]

[witjoh]

I will be picking up #677 shortly since we need this working properly for work. Feel free to add comments to the PR of any issue/wishes you have ...

[rogermartensson]

Will try to find some time for testing a new install. My problems have been around new clusters not getting initialized.

Last time I tried setting up a new cluster was before christmas (i think). I could use this branch to install and configure the nodes but I needed to initialize the cluster manually. Afterwards it worked like a charm. Could create databases and users.

If puppet isn't able to initialize clusters in Ubuntu then a workaround could be added to the documentation so you could merge the branch. (If that is acceptable)

[rogermartensson]

I tried to set up a new monbodb cluster. Latest pull-request had some issues but I left a comment in the PR.

There is still problems with Ubuntu 22.04 (jammy) to set up a new cluster. It doesn't seem to initialize it. The workaround is to manually run rs.initiate() and everything after that seem to work quite ok.

[rogermartensson]

Did some more testing when hitting a Nil-issue when using the code in the branch witjoh-mongosh as of 2024-02-17.
This code does not seem to be part of the PR as I can see but I could've missed something.

With limited knowledge in puppet module development and ruby I tracked down the problematic code to be this in lib/provider/mongodb.rb.

    if $config['auth_mechanism'] && $config['auth_mechanism'] == 'x509'
      args.push("--authenticationDatabase '$external' --authenticationMechanism MONGODB-X509")
    end

This does not look like any of the rest of the code in the file that references the config variable.
Uncomenting removed the Nil issue.

[witjoh]

Thank for testing. Just have to finish some small projects @work, will look into this after that.. Your comments are very helpfull to get this working properly

[rogermartensson]

Thanks for working on this.. Also need it. :)

I am trying to understand how the rs.initiate() is called in the module and the only way I got it to run was to enable authentication.
I still haven't to manage to set up a running cluster this way yet. I got a few steps closer when I found the 'store_creds' parameter and set it to true.
Still the "simplest" way to do it is to use 'auth = false', pick one node as the main one and run the create_admin on that node. Then manually rs.initate() and after that do the rest of the stuff you need to do finishing off with enabling authentication.

So.. not easy. Have no idea at the moment how this could be streamlined. Maybe my lack of insight in MongoDB stops me to see the full picture.

[witjoh]

I did had it working, on redhat ... but I guess during a rebase or cleaning up the commits (which were a bit of a mess I must admit) I reverted that ... I hope to get more time next week to delve in it again ...

[rogermartensson]

Found something that doesn't have anything with cluster setup but may be a security issue. When using the "store_creds = true" the diff will be shown while applying the catalogue. Ie.. the password will be shown in the logs. So that might be a good fix to have in your PR.

[rogermartensson]

I noticed that you added tls_mode as a paramter to server.pp. Nice.
Although, since I do not use TLS at all, the default of requireTLS is a bit strict?

If not it should be documented, but I guess you already have that on your todo.

[h-haaks]

@rogermartensson could you verify if release 6.0.0 is solving this?
Regarding the issue with exposed password in logs, would be nice if you could report that as a separate issue.

When it comes to setting up replica sets, there is some integration tests in the git repo at https://github.com/voxpupuli/puppet-mongodb/blob/master/spec/acceptance/replset_spec.rb.

[rogermartensson]

Sorry for being superlate. I haven't done any large testing around this and don't think I will have time to do any deep dives currently.
If I get some time to do some better testing I'll return in another issue if I notice anything.