Cookies in selenium

Cookies :

A cookie is small piece of information stored on your computer by a website you visit.

Every time the user loads the website, the browser sends the cookie back to the server to notify the website of the user’s previous activity.

Cookies have a specific life span defined by their creators. At the end of this, a cookie becomes expired. Cookies often track information like how frequently the user visits, what are the times of visits, what banners have been clicked on, what button clicked, user preferences, items in the shopping cart, etc. This allows the site to present you with information customized to fit your needs.

Cookies are usually used to store information needed for shorter periods.

Cookies initially in those earlier stages did not receive good acceptance, since rumors said it might hack your personal data. Later, people realized that cookies are harmless, and now they are highly accepted.

Cookies are domain-specific, i.e., a domain cannot read or write to a cookie created by another domain. This is done by the browser for security purposes.

Cookies are browser-specific. Each browser stores the cookies in a different location. The cookies are browser-specific, and so a cookie created in one browser(e.g., in Google Chrome) will not be accessed by another browser(Internet Explorer/Firefox).

Some browsers limit the number of cookies stored by each domain(20 cookies). If the limit is exceeded, the new cookies will replace the old cookies.

Cookie names are case-sensitive. E.g., UserName is different than username.

Where to see Cookies :
Please follow the below steps to see cookies in chrome browser.

Let's find the cookies with the help of Dora.

dora-cookies-selenium-webdriver

  1. From the Chrome menu in the top right corner of the browser, select Settings
  2. At the expand left pane of the page, click advanced
  3. Under Privacy and Security, select Content settings tab
  4. Click Cookies
  5. On this page, we can select whether to store the Cookie or not for the websites (we can select other options).
  6. Click on the See All Cookies and site data
  7. Now select any website site (domain), one website can set multiple cookies.
  8. Select any cookies; you can see the cookie details, which may look like below.

cookies-chrome-selenium-webdriver


Cache :
The cache is a temporary storage of web page resources stored on the user’s machine for quicker loading of the web pages. When you open some websites with large pictures and videos, it might take a considerable amount of time for the website to load.

The web browser stores the site contents like the images, videos, audio, etc. on your computer, so the next time you load the same website, you will find it loads faster.

The developer decides whether to store the webpage or not; he also can set browsers header that each time to bring a fresh copy of the website.

For Your Information : I have set this site's browser cache expiry limit as 30 days, that is why when you visit this site for the first time, it takes 3-4 seconds to load, but after once the cache is set this website takes 1-2 seconds to load.

Where to see Cache :

The cache will be stored in the local system, Navigate to below path to see the cache.

Path : C:UsersuserNameAppDataLocalGoogleChromeUser DataDefaultCache

cache-selenium-webdriver


Difference between Cache and Cookie :

  • A cookie is used to store information to track different characteristics related to the user, while the cache is used to make the loading of web pages faster.
  • Cookies stores information such as user preferences, while cache will keep resource files such as audio, video, or flash files.
  • Typically, cookies expire after some time, but the cache is kept in the client's machine until they are removed manually by the user.
  • A cache is a way of handling data within a single computer. A cookie comes from a website and helps the website remember you.
  • Cookies have a limit of storage 4Kb, whereas cache has unlimited storage.
  • Cookies are useful for the website's owner, and it is never useful for the user. The cache is useful for both the website's owner and User.

The Good, Bad and Ugly about Cookies

The Good :
Websites give the pre-caution about whether site stores cookies or not, and the user also can customize the browser settings not to store the cookies, Some browsers use the cookies to minimize the login time, once a cookie is stored from an IP address.

Example : Have you ever logged to Gmail after the first-time login, if you have done that, you might have noticed that Gmail never asks you about your user name and password. Because your details are stored in Google's database.

Frankly speaking, there is no good in cookies as per my knowledge.

The Bad :
The website stores the details about the cookie in their database, so they know almost everything you don their website.

Example 1: Have you ever tried to search some product on google, if you did, you could have seen that product is displayed in the ads of the website (any website) you visit.

Example 2: The website targets you once you visit their site, so all the remaining sites will show the same site's product or similar products.

The Ugly
Example 1: If you have ever searched a product in shopping websites and if you didn't buy it they will re-target you, the ads follow you like dogs (I got you : 'this is what you said in above') but the thing is they will show it as a discount as if the product price is $50 then they will show as 50% offer on the product, after the discount the price will be $75. They will do the marketing gimmick; they will sell the product at higher prices.

Example 2: Have tried to buys movie or travel tickets ?. Consider you are searching the route for Some X location to Y location, this single query on we is enough to raise the ticket prices for that IP address. Now you visit any ticket booking sites, they will charge damn high.



I have given the below screenshot for your reference. I wanted to buy a domain name which is related to tooth about one month back, normally what we do is, we will search for the domain name in some 'Gone Daddy' website.

I didn't buy it on the website as I was writing the blog for Selenium Webdriver.

For Explaining the cookie tutorial, I have visited the 'Gone Daddy' site today ( 18 Feb 2018 ) to search for the same tooth-related domain name.

The 'Gone Daddy' was showing some price, which was not too high but I wanted to try the same with incognito(private) mode, I have compared the prices, please do have a look in the below screenshot. Private mode websites cannot use the cookies present in their database. They are looting the customers based on the cookie.

cookies-gonedaddy-selenium-webdriver

My suggestion whenever you buy something use an incognito mode browser or buy it from some IP address with which you never searched about the product in recent times

Now we are out of cookies, so let's create some cookies in selenium

Handling Cookies with Selenium Webdriver

Insight : Most of the selenium people will say that we can handle the cookies using Cookies class present in Selenium Webdriver, Trust me, this is wrong that Cookies class doesn't provide you any method to create or delete or retrieve a Cookie.

WebDriver.Options : Options Interface ( inner interface ) present in the Webdriver interface provides the methods to handle the cookies, methods like creating a cookie, deleting, retrieving a cookie in Selenium Webdriver.

Every Class which implements the Webdriver interface has to implement the Cookies handling methods.

Below are the methods present in the WebDriver.Options interface :

Whenever a class Implements a Webdriver Interface, that class must implement methods provided in the interface. For example, the FirefoxDriver class provides implementation to all the methods present in the webdriver.

Now let's see what the functions and uses of the cookie handling methods. I have explained what Cookie is; please refer to it if you get a doubt about the parameters passed to the Cookie class Constructor.

1. addCookie :

We can use the addCookie method to add a specific cookie to the browser. If the cookie's domain name is left blank, it is assumed that the cookie is meant for the domain of the current document. After adding the cookie to a page, we have to refresh the page to load the added cookies.

void addCookie(Cookie cookie);

Program for adding cookies to the browser with selenium webdriver

import java.util.concurrent.TimeUnit;
import org.openqa.selenium.Cookie;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;

public class AddCookie {
	public static void main(String[] args) {
		// set chrome driver exe path
		System.setProperty("webdriver.chrome.driver", "C:/Users/user/Pictures/chromedriver.exe");
		WebDriver driver = new ChromeDriver();
		driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
		driver.get("https://google.com");

		// set the name and value for the cookie
		Cookie coo = new Cookie("karthiQ", "chercher.tech");

		// add the cookie
		driver.manage().addCookie(coo);

		// please donot write the code to close the browser.
	}
}

Take Dora's help to find our cookie; here, we may have other cookies set by Google as well along with our cookie; if we open our cookies, it may look like the below screenshot.

If you take a look at domain value '/', which means it points to the current domain as we did not set any domain, it took the current domain as the default domain.

We can set all the values in the below screenshot; for that, please refer to the Cookie class present after this section.

add-cookies-selenium-webdriver

2. deleteCookieNamed :

Delete a cookie based on its name from the current domain. This is the same as setting the named cookie's expiry date to sometime in the past when the expiry date is crossed browser automatically removes the cookie from the browser.

void deleteCookieNamed(String name);

Below code adds and removes the karthiQ named cookie.
Program for deleting named cookie from the browser with selenium webdriver

public class DeleteNamesCookie {
	public static void main(String[] args) {
		// set chrome driver exe path
		System.setProperty("webdriver.chrome.driver", "C:/Users/user/Pictures/chromedriver.exe");
		WebDriver driver = new ChromeDriver();
		driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
		driver.get("https://google.com");

		// set the name and value for the cookie
		Cookie coo = new Cookie("karthiQ", "chercher.tech");

		// set the name and value for the cookie
		Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");

		// add the cookies
		driver.manage().addCookie(coo);
		driver.manage().addCookie(ren);

		// delete the 'karthiQ' cookie
		driver.manage().deleteCookieNamed("karthiQ");

		// please donot write the code to close the browser.
	}
}

delete-named-cookie-selenium-webdriver

3. deleteCookie :

Delete a cookie from the browser's "cookie jar" using cookie's name, the domain of the cookie will be ignored. A CookieJar helps to load the cookies from the file on the local system, also CookieJar helps to save the cookies to the local system.

void deleteCookie(Cookie cookie);

Below code adds and removes the coo cookie.
Program for deleting the cookie from the browser with selenium

public static void main(String[] args) {
	// set chrome driver exe path
	System.setProperty("webdriver.chrome.driver", "C:/Users/user/Pictures/chromedriver.exe");
	WebDriver driver = new ChromeDriver();
	driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
	driver.get("https://google.com");

	// set the name and value for the cookie
	Cookie coo = new Cookie("karthiQ", "chercher.tech");

	// set the name and value for the cookie
	Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");

	// add the cookies
	driver.manage().addCookie(coo);
	driver.manage().addCookie(ren);

	// delete the 'ren' cookie
	driver.manage().deleteCookie(ren);

	// please donot write the code to close the browser.
}

delete-cookie-selenium-webdriver

Read & Write Barcode using Selenium Webdriver

4. deleteAllCookies :

Delete all the cookies for the current domain
Warning : If you delete all cookies before completely loading the page, the browser may create a cookie till the loading completes.

So there is a chance that you may be left with some cookies, so better don't delete cookies while loading. Channel ID is not a cookie, so you cannot delete them.

I have added a sleep of 10 seconds to make the webpage to load completely.

void deleteAllCookies();

Program for deleting All cookie from the browser with google.co.in domain name

// set the name and value for the cookie
Cookie coo = new Cookie("karthiQ", "chercher.tech");

// set the name and value for the cookie
Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");

// add the cookies
driver.manage().addCookie(coo);
driver.manage().addCookie(ren);

// added the sleep to allow the domain to set all the cookies
Thread.sleep(10000);
// delete all the cookies
driver.manage().deleteAllCookies();

// please donot write the code to close the browser.

delete-all-cookies-selenium-webdriver

5. getCookies :

Get all the cookies for the current domain. This is the equivalent of calling "document.cookie" and parsing the result

Set getCookies();

In the below code, we have set the 'coo', 'ren' cookies, along with that we have other cookies from the browser.

This code gets all the cookies and iterates one by one, and prints the name and value.

getCookies method returns Set of Cookie so we cannot get the values using the index. Set stores the values in random position, when you try this example in your machine, you may have the position of the Cookies changed.

driver.get("https://google.com");

// set the name and value for the cookie
Cookie coo = new Cookie("karthiQ", "chercher.tech");

// set the name and value for the cookie
Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");

// add the cookies
driver.manage().addCookie(coo);
driver.manage().addCookie(ren);

// get the cookies from the browser,
// not only cookies we set but also the cookies set by the website
Set abc = driver.manage().getCookies();
System.out.println("Number of Cookies present : "+abc.size());
for (Cookie cookie : abc) {
	System.out.println("---------------");
	System.out.println("Cookie Name : "+cookie.getName());
	System.out.println("Cookie Value : "+cookie.getValue());
}

6. getCookieNamed :

getCookieNamed returns a cookie based on the name of the cookie we pass, getCookieNamed returns null if there are no matching cookies for the name

Cookie getCookieNamed(String name);

Program for getting named('karthiQ') cookie from the browser with google.co.in domain name

driver.get("https://google.com");

// set the name and value for the cookie
Cookie coo = new Cookie("karthiQ", "chercher.tech");

// set the name and value for the cookie
Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");

// add the cookies
driver.manage().addCookie(coo);
driver.manage().addCookie(ren);

Cookie cookie = driver.manage().getCookieNamed("karthiQ");
System.out.println("---------------");
System.out.println("Cookie Name : "+cookie.getName());
System.out.println("Cookie Value : "+cookie.getValue());

get-named-cookie-selenium-webdriver

Get Components of Cookies Using Selenium Webdriver

Cookie class present in selenium webdriver helps the user to retrieve values from the Cookies, but it will not help the user to get a cookie.

Cookie class has multiple constructors; A class can have Multiple constructors by overloading the constructor.

The overloaded constructor will accept either different data types of parameters or a different number of parameters; Constructor is similar to the method constructor.

Let's learn about Cookie class constructors:


Cookie(String name, String value) : This constructor accepts only name and value, it will create a cookie which will have name and value, remaining values will be set to their default values.

The below code will set the cookie name to "selenium-webdriver.com" and its value to "Renamed to chercher.tech"

// set the name and value for the cookie
Cookie ren = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");


Cookie(String name, String value, String path) : this Constructor creates a cookie with given Name, Value, and Path. The remaining value will be set to default.

// set the name and value for the cookie
Cookie ren = new Cookie("karthiQ", "Owner", "/java/index-selenium-webdriver");


Cookie(String name, String value, String domain, String path, Date expiry) : This constructor will set Cookie with given details like Name, value, path, expiry date, remaining values will be set to their default.

Cookie cooParam5 = new Cookie("karthiQ2", "author", "www.google.co.in", "/", currentTime);


Cookie(String name, String value, String domain, String path, Date expiry) : This constructor sets the details of Cookie along with domain name.

Date currentTime = new Date(System.currentTimeMillis());
// we expiry date is in milliseconds ant it starts somewhere during 1970's
// so we will not be able to calculate it, for that purpose I took help of System class
// and used the same time for expiration
Cookie cooParam4 = new Cookie("karthiQ1", "author", "/", currentTime);


Cookie(String name, String value, String domain, String path, Date expiry, boolean isSecure) : This Constructor creates a Cookie with name, domain, path, domain, expiry date, and a secured cookie or not based on the user inputs.

Cookie cooParam6 = new Cookie("karthiQ3", "author", "www.google.co.in", "/", currentTime, true);


Cookie(String name, String value, String domain, String path, Date expiry, boolean isSecure, boolean isHttpOnly)
Create a cookie with given details, but this Constructor creates the Cookie with all the details.

Cookie cooParam7 = new Cookie("karthiQ4", "author", null, "/", currentTime, false, true);

Complete program for the different constructor of Cookies in Selenium

public static void main(String[] args) throws InterruptedException {
	// set chrome driver exe path
	System.setProperty("webdriver.chrome.driver", "C:/Users/user/Pictures/chromedriver.exe");
	WebDriver driver = new ChromeDriver();
	driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
	driver.get("https://google.com");

	// set the name and value for the cookie
	Cookie cooParam2 = new Cookie("selenium-webdriver.com", "Renamed to chercher.tech");
	Cookie cooParam3 = new Cookie("karthiQ", "Owner");

	Date currentTime = new Date(System.currentTimeMillis());
	Cookie cooParam4 = new Cookie("karthiQ1", "author", "/", currentTime);
	Cookie cooParam5 = new Cookie("karthiQ2", "author", "www.google.co.in", "/", currentTime);
	Cookie cooParam6 = new Cookie("karthiQ3", "author", "www.google.co.in", "/", currentTime, true);
	Cookie cooParam7 = new Cookie("karthiQ4", "author", null, "/", currentTime, false, true);

	// add the cookies
	driver.manage().addCookie(cooParam2);
	driver.manage().addCookie(cooParam3);
	driver.manage().addCookie(cooParam4);
	driver.manage().addCookie(cooParam5);
	driver.manage().addCookie(cooParam6);
	driver.manage().addCookie(cooParam7);

	Set abc = driver.manage().getCookies();
	System.out.println("Number of Cookies present : "+abc.size());
	for (Cookie cookie : abc) {
		System.out.println("---------------");
		System.out.println("Cookie Name : "+cookie.getName());
		System.out.println("Cookie Value : "+cookie.getValue());
		System.out.println("Cookie path : "+cookie.getPath());
		System.out.println("Cookie expiry : "+cookie.getExpiry());
		System.out.println("Cookie domain : "+cookie.getDomain());
		System.out.println("Cookie isSecure : "+cookie.isSecure());
		System.out.println("Cookie isHttpOnly : "+cookie.isHttpOnly());
	}
}

Output :

out-error-cookies-selenium-webdriver

After executing the above program, I have expected 6 cookies to be present in the browser, but unfortunately, selenium has not created 5 Cookies, Which contain more details than "name", "Value".

The cookies which had only "name", "Value" are created in the browser as well as in the output of our program.

cookie-load-failure-selenium-webdriver

Domain less Cookie : In selenium webdriver, we have to set the cookie after navigating to a domain, but if we set the cookie before navigating to a domain, then it is called as Domain less cookie.

Components of Cookies :

There are few properties for the Cookie few are mandatory, and few are optional, Let's discuss the Cookie's components and how to retrieve them.

cookies-chrome-selenium-webdriver
In Cookie class, we have few Methods which will help us to retrieve the values of the Cookie's components, the methods present in the Cookie class is similar to the components of the Cookie.

public String getName()
public String getValue()
public String getDomain()
public String getPath()
public boolean isSecure()
public boolean isHttpOnly()
public Date getExpiry()
public void validate()


Name (mandatory) : This value contains the name of the cookies. We can fetch the name of the Cookie using the getName method present in the Cookie class.

Content (mandatory) : The value of a cookie will be stored in the content component of the cookie, we can retrieve these values using getValue from the Cookie class.

Domain : The domain name allowed to access the cookie, basically the website which sets the cookie. We can retrieve the domain value using the getDomain method.

Path : The server path on which the cookie will be applied. If Path is set to '/java/', the cookie will only be available within the java directory and all sub-directories of the domain. The default value is the entire domain, and / sign is used for default. getPath gives the details about the path of the cookie

Send for : Decides like with connection (HTTP or HTTPS or both) to send the cookies Indicates that the cookie should only be transmitted over a secure HTTPS connection. When set to true, the cookie will only be set if a secure connection exists. We can get the value of the Send for a component using the isSecure method, return either true or false.

Accessible to the script : Whether to let the javascript to access the cookies or not, if it is set to no, javascript cannot access the cookie only HTTP/HTTPS can access the cookie. isHttpOnly gives details about the Accessible for value, return either true or false

Created : When the cookie got created on to the browser from the server. There is no method to retrieve this value.

Expires : The time when the cookie will expire. The values mentioned are in seconds. A value of 0 is equivalent to a 'session cookie': i.e., the cookie expires when the browser is closed, the getExpiry method fetches the value of the Expiry Date for the cookie, the return type of the getExpiry() method is Data object.

session cookie : Also called a transient cookie, a cookie that is erased when you close the Web browser. The session cookie is stored in temporary memory and is not retained after the browser is closed.

persistent cookie : Also called a permanent cookie or a stored cookie, a cookie that is stored on your hard drive until it expires (persistent cookies are set with expiration dates) or until the user deletes the cookie.

public static void main(String[] args) throws InterruptedException {
	// set chrome driver exe path
	System.setProperty("webdriver.chrome.driver", "C:/Users/user/Pictures/chromedriver.exe");
	WebDriver driver = new ChromeDriver();
	driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
	driver.get("https://google.com");

	// set the name and value for the cookie
	Cookie cooParam2 = new Cookie("q", "chercher.tech");

	// add the cookies
	driver.manage().addCookie(cooParam2);
	Cookie cookie = driver.manage().getCookieNamed("q");

	System.out.println("Cookie Name : "+cookie.getName());
	System.out.println("Cookie Value : "+cookie.getValue());
	System.out.println("Cookie path : "+cookie.getPath());
	System.out.println("Cookie expiry : "+cookie.getExpiry());
	System.out.println("Cookie domain : "+cookie.getDomain());
	System.out.println("Cookie isSecure : "+cookie.isSecure());
	System.out.println("Cookie isHttpOnly : "+cookie.isHttpOnly());
}


Valid Cookie :

  • Name of the cookies should not be null or not empty string ("") also name cannot contain ';', ':' (colon, semi-colon)
  • Value of the should not be null
  • The Expiration date can be null
  • The Path can be null or empty String, if Path is a null or empty string, by default webdriver, set the Cookie's path value to '/'
  • Domain cannot contain ':' (semi-colon)


Insight : path also cannot be null, when we use the Constructor with Name, Value, We indirectly passing a null value as we are not passing path value. But internally, the Constructor with name and Value sets the path to '/'.

How to check Whether Cookie is valid or not in Selenium :
validate() method in Cookie's class verifies whether a Cookie is valid or not, validate method will not return any value if the Cookie is valid, but it throws IllegalArgumentException when the name, domain values are not correct.

public static void main(String[] args) throws InterruptedException {
		// set chrome driver exe path
		System.setProperty("webdriver.chrome.driver", "C:/PATH/chromedriver.exe");
		WebDriver driver = new ChromeDriver();
		driver.manage().timeouts().implicitlyWait(200, TimeUnit.SECONDS);
		driver.get("https://google.com");

		// set the name and value for the cookie
		Cookie cooParam2 = new Cookie("q", "chercher.tech");

		// add the cookies
		driver.manage().addCookie(cooParam2);

		// store the cookie in Cookie class type variable
		Cookie cookie = driver.manage().getCookieNamed("q");

		// verify whether the cookie is valid or not, if valid nothing is returned1234
Comment / Suggestion Section
Point our Mistakes and Post Your Suggestions
  • MK
    Hi KarthiQ,
    
    as per your examples for adding cookie , can it be expected  that using selenium we can add cookies only with name and value . if it is true then why it is so?
    Is there any turn around for the same.
    
    Regards,
    MK
    Reply
  • Siva
    hi Karthik,
    
    I cant able to navigate to the sub topics listed on the left side of page. Kindly guide me
    Reply
    • karthiQ [admin]
      Hi Siva,
      
      Please let us know what device you are using Mobile / Desktop / laptop / tablet ?
      Reply