A_010001001110-CSDN博客

原创基础向-Pandas数据分析的常用操作（1）

获取和了解数据导入必要的库import pandas as pdimport numpy as npimport json创建随机数字构成的DataFramedata = '{"\\u6b3e\\u53f7\\u7f16\\u7801":{"0":11059080069,"1":11059080070,"2":11059080070,"3":11059080071,"4":11059080071,"5":11059080071,"6":11059081050,"7":11059081050,"

2020-07-05 13:32:14 250

原创 2020-05-20

import requests,re#requests模块抓取网页，re提取字符from pyquery import PyQuery as pq#pyquery分析HTMLimport pandas as pdfrom sqlalchemy import create_engine#pandas配合sqlalchemy将形成的DataFrame输入数据库，以便后期可视化分析BHCheaders = { 'authority': 'www.bhc520.me', 'method'

2020-05-11 21:04:11 9712

原创 Selenium批量下载抖音无水印高清视频

直接上代码，原创转载请告知import re,os,timefrom selenium import webdriverfrom selenium.webdriver.common.by import Byfrom selenium.webdriver.support.ui import WebDriverWaitfrom selenium.webdriver.support impor...

2019-12-22 18:19:23 1596 1

原创 Mysql以其他字段的截取部分更新另一字段

UPDATE ReturnRate_SKUSET codenumber = (case WHEN 款号 LIKE "%" THEN right(left(款号,7),2)ELSE NULL END)

2019-11-29 08:53:38 739

原创 Mysql创建视图语句

create view V_Cate_FourCate as select Cate2.品类, ReturnRate_SKU.月份, sum(ReturnRate_SKU.出库量) as '出库量', sum(ReturnRate_SKU.出库额) as '出库额', sum(ReturnRate_SKU.出库吊牌额) as '出库吊牌额', sum(ReturnRa...

2019-11-29 08:52:34 1978

原创 Pysql插入Excel格式的数据及查询Sql

Pysql插入EXCEL格式的数据from pymysql import *import pandas as pd from sqlalchemy import create_enginedef InsertSQL(excel_name): df = pd.read_excel(excel_name) print('Start insert!'+excel_name) conne...

2019-11-27 20:32:50 215

原创 MongoDB的LookUp及数据类型转换

链表查询，Cata_Lite时子表，中类号是目标的目标字段，中类编号是子表的目标字段，asinventory_docs输出。db.ReturnRate_SPU.aggregate([ { $lookup: { 'from': "Cate2", 'localField': "中类号", 'foreignField'...

2019-11-27 20:29:57 962

原创 Mysql数据库的Sql语句笔记

login Mysql Client under Ubuntu 18.04sudo mysql -uroot -p--loginexit/quit/contrl+d--退出select version();--查看版本select now();--查看时间数据库的基本操作show databases;--显示所有数据库create database Example charset ...

2019-11-26 00:13:34 198

原创 Ubuntu 18.4安装Mysql5.7

1.安装Mysqlsudo apt-get install Mysql-serversudo mysql-secure_installation#配置项，1选择N，2填写名为root的数据库密码，3选择N，4选择N，5选择N，6选择Y2.检查数据库状态systemctl status mysql.service#检查数据库状态3.创建数据库Studentcreate databas...

2019-11-24 19:13:03 123

原创 MongoDB的增删改查

进入MongoDB的交互界面：mongo数据库操作增：use try1#使用名为try1的库，若无，则新建名为try1的库删：db.try1.drop()#删除try1库下名为try1的collection查：show dbs#展示所有的databaseshow collections#展示try1库下所有的collections集合操作：增：db.try1.in...

2019-11-06 22:47:19 100

原创 Centos下安装MongoDB

1、创建仓库文件:vi /etc/yum.repos.d/mongodb-org-3.4.repo然后复制下面配置,保存退出[mongodb-org-3.4]name=MongoDB Repositorybaseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.4/x86_64/gpgcheck=1e...

2019-11-06 21:21:12 161

原创 Python从Mongodb获取URL下载至本地

# -*- coding: utf-8 -*-import pandas as pd import pymongoimport requestsimport timeimport randomimport numpy as npimport osclient=pymongo.MongoClient(host='localhost',port=27017)table=clien...

2019-11-06 18:54:13 279

原创 python爬取umei网17万美女图片

直接上代码from pyquery import PyQuery as pqimport reimport pymongoimport threadingclient=pymongo.MongoClient(host='localhost',port=27017)table=client.taobao.meidef save_to_mongo(result): try: i...

2019-11-01 15:20:26 1476

原创爬取天气信息并邮件发送

直接上代码#!/usr/bin/env Python#coding=UTF-8from urllib.request import urlopenfrom pyquery import PyQuery as pqimport smtplibfrom email.mime.text import MIMEText import codecsimport timedef sente...

2019-11-01 15:17:16 1829

原创 Python爬取淘宝网商品信息

直接上代码#!/usr/bin/env Python#coding=UTF-8import timeimport pymongofrom pyquery import PyQuery as pqfrom selenium import webdriverfrom selenium.webdriver.common.by import Byfrom selenium.webdrive...

2019-11-01 15:12:28 1229

原创 Python多线程爬取7160网站美女图片

直接上代码#coding=UTF-8from urllib.request import urlopenfrom pyquery import PyQuery as pqimport reimport pymongoimport threadingclient=pymongo.MongoClient(host='111.231.193.15',port=27017)table=cl...

2019-11-01 14:56:25 37468

A_010001001110的博客