visit
I was working on a NodeJS/Express application for practice and I remembered I pushed the .env
file to my remote repository.
Using the scripts below, I was able to generate repositories whose code included mongodb+srv:
// index.ts
import dotenv from "dotenv"
dotenv.config()
import axios from "axios";
import fs from "fs/promises";
import cliProgress from "cli-progress";
const jsonpath = "list_of_repo.json";
const makeSearch = async (page: number) => {
const config = {
headers: {
Authorization: `Token ${process.env.GITHUB_API_TOKEN}`,
},
};
const url = `//api.github.com/search/code?q=mongodb%2Bsrv+in:file&page=${page}&per_page=100`;
const result: {
items: { html_url: string; repository: { html_url: string } }[];
} = await axios.get(url, config);
// make an an object from result
let obj = {};
result.data.items.forEach((item) => {
obj[item.repository.html_url] = item.html_url;
});
await addToJson(jsonpath, obj);
};
async function addToJson(jsonpath: string, data?: object) {
const oldJson = (await fs.readFile(jsonpath)).toString();
let jsonData = JSON.stringify(data, null, 2);
if (oldJson) {
jsonData = JSON.stringify(
{ ...JSON.parse(oldJson), ...JSON.parse(jsonData) },
null,
2
);
}
await fs.writeFile(jsonpath, jsonData);
}
async function main() {
// I included a CLI progress loader because, who doesn’t like a loader.
const bar1 = new cliProgress.SingleBar(
{},
cliProgress.Presets.shades_classic
);
// number of iteration 10
bar1.start(10, 0);
for (let i = 1; i <= 10; i++) {
await makeSearch(i);
bar1.update(1);
}
bar1.stop();
}
main();
The results provided does not mean that an actual MongoDB connection string exists, it only implies that the repositories in the result have an in-file code that matches mongodb+srv:
Some of the results include old commits in the commit history: Just like my mistake that led to this article, sometimes we forget to create a .gitignore
file at the beginning of a project and have some secrets staged somewhere in the commit history.
Some results included messages from different log files and environment files: This probably happened due to not including a .gitignore.
GitHub provides a with numerous types of .gitignore
templates for different languages, frameworks, tools, IDE e.t.c.
And I created a simple interactive CLI to generate .gitignore
templates based on the GitHub lists.
You can find the Interactive CLI tool to generate your .gitignore
templates here:
Previously published