Everything depends on the plugin/lib developer approach. Typically gradle. properties file is used to store secrets. Alternatively some json/xml files.
There are only few plugins that use env vars and I think there is no easy way to keep those env vars project specific.

I think it's a good solution.
Alternatively you can create new debug configuration (see screenshot), add breakpoints, run from console:

./gradlew build --stacktrace -D org.gradle.debug=true -D org.gradle.daemon=false

and run your newly created configuration.