+91 7387620624

Removing Duplicates

Introduction

One of the most useful features of a Set is its ability to automatically remove duplicate values. Since a Set stores only unique values, it is commonly used to eliminate duplicates from arrays and other collections.

Removing duplicates is a frequent task in web development, Node.js applications, data processing, API testing, and automation frameworks such as Selenium, Playwright, and Cypress.

For automation engineers, removing duplicates is useful when working with test data, browser names, user IDs, API endpoints, environment names, execution results, and any collection where repeated values should be eliminated.

Why Remove Duplicates?

Duplicate values can cause problems such as:

  • Repeated test execution.

  • Duplicate API requests.

  • Incorrect reports.

  • Redundant browser configurations.

  • Data inconsistency.

  • Increased processing time.

Using a Set provides a simple and efficient solution.

How Does a Set Remove Duplicates?

When duplicate values are added to a Set, JavaScript automatically keeps only the first occurrence and ignores the rest.

Example

let numbers = new Set([

    10,

    20,

    20,

    30,

    10

]);

console.log(numbers);

Output

Set(3) { 10, 20, 30 }

The duplicate values 10 and 20 are removed automatically.

Removing Duplicates from an Array

The most common technique is to convert an array into a Set.

Example

let numbers = [

    1,

    2,

    2,

    3,

    4,

    4,

    5

];

let uniqueNumbers = new Set(numbers);

console.log(uniqueNumbers);

Output

Set(5) { 1, 2, 3, 4, 5 }

Converting the Set Back to an Array

In many applications, you may need the result as an array instead of a Set.

Use the spread operator (...).

let numbers = [

    1,

    2,

    2,

    3,

    4,

    4

];

let uniqueNumbers = [...new Set(numbers)];

console.log(uniqueNumbers);

Output

[1, 2, 3, 4]

This is the most popular JavaScript technique for removing duplicates from an array.

Removing Duplicate Strings

let fruits = [

    "Apple",

    "Mango",

    "Apple",

    "Orange",

    "Mango"

];

let uniqueFruits = [...new Set(fruits)];

console.log(uniqueFruits);

Output

["Apple", "Mango", "Orange"]

Removing Duplicate Boolean Values

let values = [

    true,

    false,

    true,

    false

];

let uniqueValues = [...new Set(values)];

console.log(uniqueValues);

Output

[true, false]

Real-World Example

Suppose an application stores browser names.

let browsers = [

    "Chrome",

    "Firefox",

    "Chrome",

    "Edge",

    "Firefox"

];

let uniqueBrowsers = [...new Set(browsers)];

console.log(uniqueBrowsers);

Output

["Chrome", "Firefox", "Edge"]

Another example:

Remove duplicate departments.

let departments = [

    "QA",

    "HR",

    "QA",

    "Development"

];

let uniqueDepartments = [...new Set(departments)];

console.log(uniqueDepartments);

Output

["QA", "HR", "Development"]

Automation Testing Example

Automation engineers often remove duplicate data before executing automated tests.

Playwright Example

Remove duplicate browser names.

const browsers = [

    "chromium",

    "firefox",

    "chromium"

];

const uniqueBrowsers = [...new Set(browsers)];

console.log(uniqueBrowsers);

Output

["chromium", "firefox"]

Selenium Example

Remove duplicate browser configurations.

const browserNames = [

    "chrome",

    "chrome",

    "firefox"

];

console.log([...new Set(browserNames)]);

Output

["chrome", "firefox"]

Cypress Example

Remove duplicate environments.

const environments = [

    "QA",

    "QA",

    "UAT",

    "Production"

];

console.log([...new Set(environments)]);

Output

["QA", "UAT", "Production"]

API Testing Example

Remove duplicate API endpoints.

const endpoints = [

    "/users",

    "/orders",

    "/users",

    "/products"

];

console.log([...new Set(endpoints)]);

Output

["/users", "/orders", "/products"]

Data-Driven Testing Example

Remove duplicate usernames.

const usernames = [

    "admin",

    "tester",

    "admin",

    "guest"

];

console.log([...new Set(usernames)]);

Output

["admin", "tester", "guest"]

Common Mistakes

Expecting a Set to Return an Array

let numbers = new Set([

    1,

    2,

    2

]);

console.log(numbers);

Output

Set(2) { 1, 2 }

If you need an array, convert it using the spread operator.

let array = [...numbers];

Forgetting the Spread Operator

Incorrect:

let uniqueNumbers = new Set(numbers);

This creates a Set.

Correct:

let uniqueNumbers = [...new Set(numbers)];

This creates an array without duplicates.

Expecting Duplicate Objects to Be Removed

let users = [

    { name: "John" },

    { name: "John" }

];

console.log([...new Set(users)]);

Output

[
  { name: "John" },
  { name: "John" }
]

Although the objects contain the same data, they are different object references, so both remain in the Set.

Best Practices

Use Sets for Duplicate Removal

Using a Set is the simplest and most efficient way to remove duplicate primitive values from an array.

Convert Back to an Array When Needed

If the rest of your application expects an array, use the spread operator after creating the Set.

Use Meaningful Variable Names

Instead of:

let s = [...new Set(arr)];

Use:

let uniqueUsers = [...new Set(users)];

This improves readability.

Be Careful with Objects

Sets remove duplicate primitive values (such as numbers, strings, and booleans) easily. For objects and arrays, duplicates are determined by reference, not by content.

Conclusion

Removing duplicates is one of the most practical uses of JavaScript Sets. By converting an array into a Set, duplicate values are automatically removed, resulting in a collection of unique values. If needed, the Set can be converted back into an array using the spread operator.

For automation engineers, this technique is extremely valuable when cleaning test data, browser lists, API endpoints, user IDs, environment names, and execution results. Mastering duplicate removal with Sets helps create cleaner, more efficient, and reliable JavaScript applications and automation scripts.

Frequently Asked Questions (FAQs)

How does a Set remove duplicates?

A Set automatically stores only unique values and ignores repeated ones.

How do you remove duplicates from an array?

let uniqueArray = [...new Set(array)];

Can a Set remove duplicate strings?

Yes.

let uniqueFruits = [...new Set(fruits)];

Can a Set remove duplicate numbers?

Yes.

let uniqueNumbers = [...new Set(numbers)];

Does a Set remove duplicate objects?

No. Objects are compared by reference, not by their content.

Why is removing duplicates important in automation testing?

Automation engineers remove duplicates to prevent repeated test execution, eliminate duplicate API requests, clean browser lists, manage unique user IDs, and improve the accuracy and efficiency of automated test suites.

Key Takeaways

  • A Set stores only unique values.

  • Duplicate values are automatically removed.

  • Converting an array to a Set removes duplicates.

  • Use the spread operator (...) to convert a Set back into an array.

  • This technique works well for numbers, strings, and booleans.

  • Objects are compared by reference, not by content.

  • Removing duplicates helps maintain clean and consistent data.

  • It is widely used in JavaScript development and automation testing.

  • The combination [...]new Set(array) is the most common approach for duplicate removal.

  • Mastering duplicate removal with Sets improves code quality and application performance.