推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
thekoc
V2EX  ›  Python

Python 多线程爬虫如何优雅的退出?

  •  
  •   thekoc · Nov 14, 2016 · 4887 views
    This topic created in 3603 days ago, the information mentioned may be changed or developed.

    爬虫如果不退出就会一直爬的……但是每次按 ctrl+c 退出的时候似乎都有点问题。爬虫的退出如何正确的实现?

    8 replies  •  2016-11-24 15:36:16 +08:00
    cdwyd
        1
    cdwyd  
       Nov 14, 2016 via Android
    标记下看看有什么好的解决方法没
    wellsc
        2
    wellsc  
       Nov 14, 2016 via iPhone
    sudo pkill -9 python
    loalj
        3
    loalj  
       Nov 14, 2016
    t = threading.Thread()
    t.daemon = True
    em70
        4
    em70  
       Nov 14, 2016 via iPhone
    每爬一个数据检测指定目录下指定文件里的值, 1 就忽略, 0 就退出,开另一个窗口用 vi 控制这个文件的值
    Thoxvi
        5
    Thoxvi  
       Nov 14, 2016 via Android
    看需求吧…生产者爬到阈值丢一个毒丸到队列消费者就没了
    stamaimer
        6
    stamaimer  
       Nov 14, 2016 via iPhone
    不知道为啥不能优雅的退出
    ryd994
        7
    ryd994  
       Nov 15, 2016
    用 signal 模块,保证信号发到主线程
    设置子线程 daemon=True
    同时,其他线程捕捉 keyboardinterrput , 然后 interrupt_main https://docs.python.org/2/library/thread.html#thread.interrupt_main
    Powered
        8
    Powered  
       Nov 24, 2016 via iPhone
    调度系统…
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Privacy   ·   Solana   ·   2711 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 39ms · UTC 07:03 · PVG 15:03 · LAX 00:03 · JFK 03:03
    ♥ Do have faith in what you're doing.