[dlmarion]

This commit adds two components that are designed to be used together. The first is the ManagerStress component that creates a configurable number of Accumulo clients to drive a weighted workload of table operations, including create, delete, split, merge, availability changes, compactions, and bulk imports.

The second component is the StressCompactor, which does not perform a compaction, but returns success, failure, or cancellation based on a weighted criteria. It starts a configurable number of Compactor instances in a single JVM, each running in their own thread and with their own Thrift RPC server.

The bin/manager-stress/configure-mgr-stress-test.sh script is the main entrypoint for the user. It prompts the user for information and outputs commands to be run by the user to start the StressCompactor's and ManagerStress test.

Before using this feature the user will need to create a directory in HDFS that will be used by the workers to create RFiles for the bulk import simulation and create a Resource Group for the compactors.

Assisted-By: OpenAI GPT-6 Luna

[DomGarguilo]

Tried this out and here are my initial thoughts in no particular order:

1. I looked for something to run in /bin dir since thats where every other test script lives. Having this in a new subdir breaks that convention. Wondering if we can align with convention here so we could just run something like bin/mstress (or similar)
2. The long args list for the run command is cumbersome. Also, other tests read a props file instead. If we use that path thats already built, I think it would cut down on a lot of the new code that parses the args. I think the current approach with a bunch of args attached is more explicit but I'm not sure its worth that. Is there a reason you didn't use a props file like the other tests?
3. I had trouble stopping the stress test. ctrl + C was not working and I had to ctrl + Z which left some orphaned JVMs

I have not really looked at the code much yet so these are just observations from using the new tool

[dlmarion]

2. The long args list for the run command is cumbersome. Also, other tests read a props file instead. If we use that path thats already built, I think it would cut down on a lot of the new code that parses the args. I think the current approach with a bunch of args attached is more explicit but I'm not sure its worth that. Is there a reason you didn't use a props file like the other tests?

Just curious, did you start with the configure-mgr-stress-test.sh script? It takes your input and generates commands that you can copy/paste.

[dlmarion]

Is there a reason you didn't use a props file like the other tests?

Regarding this question - I could look at this, but I was envisioning multiple ManagerStress and StressCompactor JVMs being started, potentially with different configurations. Depending on the configuration the JAVA_OPTS will need to be specified for different JVM sizes to deal with the number of objects in use concurrently.

[DomGarguilo]

Just curious, did you start with the configure-mgr-stress-test.sh script? It takes your input and generates commands that you can copy/paste.

I did not use it initially but just tried it. Seems to work okay. Noticed the value for the config file arg points to a file that doesn't exist by default:

  --config-file /home/dom/github/accumulo-testing/bin/manager-stress/../../conf/accumulo-client.properties

This points to my accumulo-testing/conf/accumulo-client.properties ultimately but that file is not there. Also the /../../ was a big confusing at first and maybe could be simplified if we stick with this approach.

[DomGarguilo]

Is there a reason you didn't use a props file like the other tests?

Regarding this question - I could look at this, but I was envisioning multiple ManagerStress and StressCompactor JVMs being started, potentially with different configurations. Depending on the configuration the JAVA_OPTS will need to be specified for different JVM sizes to deal with the number of objects in use concurrently.

I think props can still cover that right? The JAVA_OPTS are just env vars and will work the same either way right? For example JAVA_OPTS=-Xmx16g bin/mstress run. Each configuration could have its own props file and use -o overrides like cingest does. That would let us drop the configure script and Options parsing classes