#!/bin/bash while read URL do echo $URL echo $URL/ done | grep -v "//$" | sort | uniq