因此，我做一些数据刮与 selenium 使用这个网站，https://uk.trustpilot.com/，我导航我的方式接受cookie，导航到查看所有按钮，以查看类别和子类别的列表，通过使用下面的方法，我做了一个字典，具有以下结构，这是由get_catogories调用返回：

key = category name
value = url, sub_cat_dictionary
sub_cat_dictionary key = sub category name
sub_cat_dictionary value = url
dictionary['key']

然后，我允许用户选择他们想要从哪个类别提取信息，或者他们想要从哪个子类别提取信息，我实现了下面的方法：

def select_cat_and_or_subcat(self):
        """
        Allows the user to select which catorgy and sub catogory where they would like to begin extracting data from
        """
        #gets the data of catogories and sub_cat
        catogories_and_sub_cat = self.get_catogories()
        
        
        keys = catogories_and_sub_cat.keys()
        #presents the catogory and subcat to the user
        for key in keys:
            print('---------------------------------------')
            print(f'Catogory: {key} ')
            _, sub_cat = catogories_and_sub_cat[f'{key}']
            for sub_key in sub_cat.keys():
                
                print(f"SubCatorgy: {sub_key}")
        try:         
            user_catogory = input('Please select which catogory you would like to explore: ')
            #uses the user input as a key and gets out the cat url and sub_cat dictionary
            cat_url, cat_sub_cat = catogories_and_sub_cat[f'{user_catogory}']
            #may like to explore and extract data from main catorgory or sub_cat
            user_sub_catorgory = input('Please select the sub-catorgory or 0 for no sub-catorgory: ')
        except:
            print('exiting....due to invalid catorgory being selected')
        try:
            
            if int(user_sub_catorgory) == 0:
                #if zero navigate to main cat url
                self.driver.get(cat_url)
                print('done')
            #else if sub_cat check if its within the cat_sub keys    
            elif user_sub_catorgory in cat_sub_cat.keys():
                sub_url_for_cat = cat_sub_cat[f'{user_sub_catorgory}']
                #load the webpage for the sub_cat
                self.driver.get(sub_url_for_cat)
        except: 
            print('exiting....due to invalid response')

还有猫和子猫

---------------------------------------
Catogory: Animals & Pets
SubCatorgy: Animal Health
SubCatorgy: Animal Parks & Zoo
SubCatorgy: Cats & Dogs
SubCatorgy: Horses & Riding
SubCatorgy: Pet Services
SubCatorgy: Pet Stores
---------------------------------------
Catogory: Beauty & Well-being
SubCatorgy: Cosmetics & Makeup
SubCatorgy: Hair Care & Styling
SubCatorgy: Personal Care
SubCatorgy: Salons & Clinics
SubCatorgy: Tattoos & Piercings
SubCatorgy: Wellness & Spa
SubCatorgy: Yoga & Meditation
---------------------------------------
Catogory: Business Services
SubCatorgy: Administration & Services
SubCatorgy: Associations & Centers
SubCatorgy: HR & Recruiting
SubCatorgy: Import & Export
SubCatorgy: IT & Communication
SubCatorgy: Office Space & Supplies
SubCatorgy: Print & Graphic Design
SubCatorgy: Research & Development
SubCatorgy: Sales & Marketing
SubCatorgy: Shipping & Logistics
SubCatorgy: Wholesale
---------------------------------------
Catogory: Construction & Manufacturing
SubCatorgy: Architects & Engineers
SubCatorgy: Building Materials
SubCatorgy: Chemicals & Plastic
SubCatorgy: Construction Services
SubCatorgy: Contractors & Consultants
SubCatorgy: Factory Equipment
SubCatorgy: Garden & Landscaping
SubCatorgy: Industrial Supplies
SubCatorgy: Manufacturing
SubCatorgy: Production Services
SubCatorgy: Tools & Equipment
---------------------------------------
Catogory: Education & Training
SubCatorgy: Colleges & Universities
SubCatorgy: Courses & Classes
SubCatorgy: Education Services
SubCatorgy: Language Learning
SubCatorgy: Music & Theater Classes
SubCatorgy: School & High School
SubCatorgy: Specials Schools
SubCatorgy: Vocational Training
---------------------------------------
Catogory: Electronics & Technology
SubCatorgy: Appliances & Electronics
SubCatorgy: Audio & Visual
SubCatorgy: Computers & Phones
SubCatorgy: Internet & Software
SubCatorgy: Repair & Services
---------------------------------------
Catogory: Events & Entertainment
SubCatorgy: Adult Entertainment
SubCatorgy: Children's Entertainment
SubCatorgy: Clubbing & Nightlife
SubCatorgy: Events & Venues
SubCatorgy: Gambling
SubCatorgy: Gaming
SubCatorgy: Museums & Exhibits
SubCatorgy: Music & Movies
SubCatorgy: Theater & Opera
SubCatorgy: Wedding & Party
---------------------------------------
Catogory: Food, Beverages & Tobacco
SubCatorgy: Agriculture & Produce
SubCatorgy: Asian Grocery Stores
SubCatorgy: Bakery & Pastry
SubCatorgy: Beer & Wine
SubCatorgy: Beverages & Liquor
SubCatorgy: Candy & Chocolate
SubCatorgy: Coffee & Tea
SubCatorgy: Food Production
SubCatorgy: Fruits & Vegetables
SubCatorgy: Grocery Stores & Markets
SubCatorgy: Lunch & Catering
SubCatorgy: Meat, Seafood & Eggs
SubCatorgy: Smoking & Tobacco
---------------------------------------
Catogory: Health & Medical
SubCatorgy: Clinics
SubCatorgy: Dental Services
SubCatorgy: Diagnostics & Testing
SubCatorgy: Doctors & Surgeons
SubCatorgy: Health Equipment
SubCatorgy: Hospital & Emergency
SubCatorgy: Medical Specialists
SubCatorgy: Mental Health
SubCatorgy: Pharmacy & Medicine
SubCatorgy: Physical Aids
SubCatorgy: Pregnancy & Children
SubCatorgy: Therapy & Senior Health
SubCatorgy: Vision & Hearing
---------------------------------------
Catogory: Hobbies & Crafts
SubCatorgy: Art & Handicraft
SubCatorgy: Astrology & Numerology
SubCatorgy: Fishing & Hunting
SubCatorgy: Hobbies
SubCatorgy: Metal, Stone & Glass Work
SubCatorgy: Music & Instruments
SubCatorgy: Needlework & Knitting
SubCatorgy: Outdoor Activities
SubCatorgy: Painting & Paper
---------------------------------------
Catogory: Home & Garden
SubCatorgy: Bathroom & Kitchen
SubCatorgy: Cultural Goods
SubCatorgy: Decoration & Interior
SubCatorgy: Energy & Heating
SubCatorgy: Fabric & Stationery
SubCatorgy: Furniture Stores
SubCatorgy: Garden & Pond
SubCatorgy: Home & Garden Services
SubCatorgy: Home Goods Stores
SubCatorgy: Home Improvements
---------------------------------------
Catogory: Home Services
SubCatorgy: Cleaning Service Providers
SubCatorgy: Craftsman
SubCatorgy: House Services
SubCatorgy: House Sitting & Security
SubCatorgy: Moving & Storage
SubCatorgy: Plumbing & Sanitation
SubCatorgy: Repair Service Providers
---------------------------------------
Catogory: Legal Services & Government
SubCatorgy: Customs & Toll
SubCatorgy: Government Department
SubCatorgy: Law Enforcement
SubCatorgy: Lawyers & Attorneys
SubCatorgy: Legal Service Providers
SubCatorgy: Libraries & Archives
SubCatorgy: Municipal Department
SubCatorgy: Registration Services
---------------------------------------
Catogory: Media & Publishing
SubCatorgy: Books & Magazines
SubCatorgy: Media & Information
SubCatorgy: Photography
SubCatorgy: Video & Sound
---------------------------------------
Catogory: Money & Insurance
SubCatorgy: Accounting & Tax
SubCatorgy: Banking & Money
SubCatorgy: Credit & Debt Services
SubCatorgy: Insurance
SubCatorgy: Investments & Wealth
SubCatorgy: Real Estate
---------------------------------------
Catogory: Public & Local Services
SubCatorgy: Employment & Career
SubCatorgy: Funeral & Memorial
SubCatorgy: Housing Associations
SubCatorgy: Kids & Family
SubCatorgy: Military & Veteran
SubCatorgy: Nature & Environment
SubCatorgy: Professional Organizations
SubCatorgy: Public Services & Welfare
SubCatorgy: Religious Institutions
SubCatorgy: Shelters & Homes
SubCatorgy: Waste Management
---------------------------------------
Catogory: Restaurants & Bars
SubCatorgy: African & Pacific Cuisine
SubCatorgy: Bars & Cafes
SubCatorgy: Chinese & Korean Cuisine
SubCatorgy: European Cuisine
SubCatorgy: General Restaurants
SubCatorgy: Japanese Cuisine
SubCatorgy: Mediterranean Cuisine
SubCatorgy: Middle Eastern Cuisine
SubCatorgy: North & South American Cuisine
SubCatorgy: Southeast Asian Cuisine
SubCatorgy: Takeaway
SubCatorgy: Vegetarian & Diet
---------------------------------------
Catogory: Shopping & Fashion
SubCatorgy: Accessories
SubCatorgy: Clothing & Underwear
SubCatorgy: Clothing Rental & Repair
SubCatorgy: Costume & Wedding
SubCatorgy: Jewelry & Watches
SubCatorgy: Malls & Marketplaces
---------------------------------------
Catogory: Sports
SubCatorgy: Ball Games
SubCatorgy: Bat-and-ball Games
SubCatorgy: Bowls & Lawn Sports
SubCatorgy: Dancing & Gymnastics
SubCatorgy: Equipment & Associations
SubCatorgy: Extreme Sports
SubCatorgy: Fitness & Weight Lifting
SubCatorgy: Golf & Ultimate
SubCatorgy: Hockey & Ice Skating
SubCatorgy: Martial arts & Wrestling
SubCatorgy: Outdoor & Winter Sports
SubCatorgy: Shooting & Target Sports
SubCatorgy: Swimming & Water Sports
SubCatorgy: Tennis & Racquet Sports
---------------------------------------
Catogory: Travel & Vacation
SubCatorgy: Accommodation & Lodging
SubCatorgy: Activities & Tours
SubCatorgy: Airlines & Air Travel
SubCatorgy: Hotels
SubCatorgy: Travel Agencies
---------------------------------------
Catogory: Utilities
SubCatorgy: Energy & Power
SubCatorgy: Oil & Fuel
SubCatorgy: Water Utilities
---------------------------------------
Catogory: Vehicles & Transportation
SubCatorgy: Air & Water Transport
SubCatorgy: Airports & Parking
SubCatorgy: Auto Parts & Wheels
SubCatorgy: Bicycles
SubCatorgy: Cars & Trucks
SubCatorgy: Motorcycle & Powersports
SubCatorgy: Other Vehicles & Trailers
SubCatorgy: Taxis & Public Transport
SubCatorgy: Vehicle Rental
SubCatorgy: Vehicle Repair & Fuel