[ZEPPELIN-6171] Resolve nested LDAP groups on FreeIPA via memberOf

[HwangRock]

What is this PR for?

LdapRealm resolves a user's groups in one of two ways today: the Active Directory LDAP_MATCHING_RULE_IN_CHAIN operator, or a group search over the member attribute. Neither works for nested groups on FreeIPA / 389 Directory Server — the AD matching rule isn't supported there, and the plain member search only sees direct members. So if a user is in dev and dev is a member of eng, the user's eng membership (and any role mapped to it) silently disappears.

This adds a third path. When ldapRealm.groupSearchEnableMemberOf = true, the realm reads the user entry's own memberOf attribute instead of walking the group tree. FreeIPA/389 DS already flattens direct and nested membership onto memberOf, so a single lookup of the user entry gives you the full set, nested groups included.

While adding it I split rolesFor() into three small per-strategy methods (matching-rule / group-membership / memberOf) so each path is readable on its own. The two existing paths are moved as-is, no behavior change.

New settings:

# resolve nested groups via the user's memberOf attribute (e.g. FreeIPA / 389 DS)
ldapRealm.groupSearchEnableMemberOf = true
ldapRealm.memberOfAttribute = memberOf

A few decisions worth calling out:

• Precedence when both are on. If groupSearchEnableMatchingRuleInChain and groupSearchEnableMemberOf are both set, the matching-rule path wins and we log a one-time warning, rather than refusing to start. This keeps existing AD setups behaving exactly as before; the cost is that a contradictory config isn't hard-rejected, just warned.
• Group name comes from the leaf RDN only. A memberOf value is a full group DN, and we take the group name from its leaf RDN without scanning ancestors. On FreeIPA a container on the path (cn=groups,cn=accounts,...) has the same cn= type as the group itself, so scanning would pick the wrong one. A malformed DN is skipped (and logged) instead of failing the login.
• No behavior change for the existing paths. The matching-rule branch still doesn't populate the session group-name set the way the default branch does — that's pre-existing and I left it alone on purpose rather than "fixing" it in a refactor.

One prerequisite (documented in shiro_authentication.md): memberOf is only returned to an authenticated bind, which Zeppelin already does via systemUsername/systemPassword. Deployments that split groups and members across separate backends may also need server-side memberOf scope configuration.

What type of PR is it?

Improvement

Todos

• memberOf-based group resolution path
• split rolesFor() into per-strategy methods
• unit tests
• docs

What is the Jira issue?

ZEPPELIN-6171

How should this be tested?

LdapRealmTest (11 tests, mock-based) covers the memberOf path: nested resolution, matching-rule precedence, leaf-RDN fallback, and the empty / missing memberOf cases. The existing tests pass unchanged, which is what pins down that the two old paths still behave the same.

I also ran it end-to-end against a real 389 Directory Server (FreeIPA's LDAP engine) with a nested setup — alice ∈ dev, and dev ∈ eng:

# alice's memberOf at the directory level (server-flattened, ground truth)
$ ldapsearch ... -b 'uid=alice,ou=people,dc=example,dc=com' -s base memberOf
dn: uid=alice,ou=people,dc=example,dc=com
memberOf: cn=dev,ou=groups,dc=example,dc=com
memberOf: cn=eng,ou=groups,dc=example,dc=com

# LdapRealm.rolesFor(alice) against that server, only the new flag differs:
case A  groupSearchEnableMemberOf=false  ->  [dev]        # nested cn=eng missing
case B  groupSearchEnableMemberOf=true   ->  [dev, eng]   # nested cn=eng resolved

So the nested group is recovered only with the new flag on.

Questions:

• Does the license files need to update? No
• Is there breaking changes for older versions? No
• Does this needs documentation? Yes — docs/setup/security/shiro_authentication.md is updated.

[tbonelee]

On FreeIPA, memberOf also holds entries that aren't groups. FreeIPA configures the MemberOf plugin with memberUser, memberHost and ipaOwner in addition to member (install/share/memberof-conf.ldif), so a user's memberOf also carries the DNs of HBAC rules, sudo rules and roles (cn=roles). The current code treats all of them as groups.

I checked this by setting up a 389 DS container with the same plugin config and DN layout as FreeIPA (cn=groups,cn=accounts, etc.) and logging in to Zeppelin built from this branch. alice ∈ dev ∈ eng, and alice also has the helpdesk role, one HBAC rule and one sudo rule:

• groupSearchEnableMemberOf=false: ["user_role"]
• groupSearchEnableMemberOf=true: ["user_role","admin_role","helpdesk","aaaa-1111","bbbb-2222"]

The nested group is resolved as intended, but the ipaUniqueIDs of the HBAC and sudo rules and the role name also became roles. If such a name matches a key in rolesByGroup, a roles[...] in [urls], or a role name used in note permissions, it grants permissions nobody intended.

How about accepting only DNs under groupSearchBase, like the other two paths? It can be checked with new LdapName(groupDn).startsWith(new LdapName(groupSearchBase)), and in the same setup the result became ["user_role","admin_role"].

When groupSearchBase isn't set, it falls back to searchBase (usually the root), so the filter wouldn't filter anything. Since this flag isn't in any release yet, how about starting strict: in memberOf mode, accept no memberOf values when groupSearchBase isn't set, and warn in onInit()? Relaxing that later stays compatible, but tightening it later would change existing users' roles.

[HwangRock]

Thanks for catching this, it's a critical one!
Fixed it to only accept DNs under groupSearchBase like you suggested.

[tbonelee]

Since the memberOf path doesn't look up the group entry, it can only get the name from the configured groupIdAttribute when the leaf RDN type matches it. When the type differs, how about skipping the value instead of using another attribute's value? For example, with groupIdAttribute = gidNumber and rolesByGroup keyed by gid numbers, turning on memberOf mode now silently adds the cn value as an unmapped role. When skipping, a DEBUG log plus a line in the docs ("in memberOf mode, the leaf RDN type of group DNs must match groupIdAttribute") would be enough.

Such a setup can still use memberOf mode by switching to groupIdAttribute = cn and keying rolesByGroup by cn. With the current code rolesByGroup has to be re-keyed by cn anyway, so the only extra requirement is that groupIdAttribute matches what is actually used. If a setup that needs groupIdAttribute read from the group entry shows up later, a lookup only for mismatched types can be added then. Skipping now means that addition won't change existing users' behavior.

This WARN also repeats on every request. In the setup from the comment above, the HBAC and sudo rule DNs went through this path, and 10 calls to the notebook API grew this WARN from 2 lines to 42. With the scope filter this path would mostly see group DNs, but a mismatched config would repeat the same way.

With this change, the first assertion in testGroupNameFromMemberOfDnFallback would expect null.

[HwangRock]

Applied this as you suggested.

[tbonelee]

Both flags are fixed at configuration time, so checking once in onInit() seems enough. That would remove the warnedBothGroupSearchModes field and the per-request check.

Also, testWarnBothGroupSearchModesLogsOnlyOnce doesn't check the log count; it only compares the roles from the two calls. It still passes after removing warnedBothGroupSearchModes = true;, which makes the warning go from 1 to 2 within this test. With the check in onInit(), "only once" is guaranteed by structure, and the precedence is already covered by testRolesForMatchingRuleInChainTakesPrecedenceOverMemberOf, so I think this test could be dropped.

[HwangRock]

Applied your suggestion. Also dropped that test you pointed out.

[tbonelee]

The method name already says this, so this comment could be removed.

Suggested change

// Default path: search groups and check the member attribute for the user DN.

[tbonelee]

Merged into master