D7net
Home
Console
Upload
information
Create File
Create Folder
About
Tools
:
/
lib
/
python3.6
/
site-packages
/
future
/
backports
/
urllib
/
__pycache__
/
Filename :
robotparser.cpython-36.opt-1.pyc
back
Copy
3 �c� � @ s� d dl mZmZmZ d dlmZ d dlmZ d dlm Z mZ e e_ ee_dgZ G dd� de�ZG dd� de�ZG d d � d e�ZdS )� )�absolute_import�division�unicode_literals)�str)�urllib)�parse�request�RobotFileParserc @ sZ e Zd ZdZddd�Zdd� Zdd� Zd d � Zdd� Zd d� Z dd� Z dd� Zdd� ZdS )r zs This class provides a set of methods to read, parse and answer questions about a single robots.txt file. � c C s, g | _ d | _d| _d| _| j|� d| _d S )NFr )�entries� default_entry�disallow_all� allow_all�set_url�last_checked)�self�url� r �!/usr/lib/python3.6/robotparser.py�__init__ s zRobotFileParser.__init__c C s | j S )z�Returns the time the robots.txt file was last fetched. This is useful for long-running web spiders that need to check for new robots.txt files periodically. )r )r r r r �mtime&