<jupyter_start><jupyter_text>本文将介绍如何利用 Arctern 处理大型地理空间数据，并使用 keplergl 进行数据可视化分析上海渣土车数据集。加载上海市渣土车运行轨迹数据，原数据共 8 列，此处只读取本文分析所需要的 4 列：车牌号、时间、经度、维度。<jupyter_code>import pandas as pd
import arctern
from arctern import GeoSeries

sh_columns=[
    ("plate_number","string"),
    ("pos_time","string"),
    ("pos_longitude","double"),
    ("pos_latitude","double"),
    ("pos_direction0","double"),
    ("pos_direction1","double"),
    ("pos_direction2","double"),
    ("pos_direction3","double")
]

sh_select_columns={
                   "plate_number",
                   "pos_time",
                   "pos_longitude",
                   "pos_latitude"
                  }

sh_schema={}
sh_use_cols=[]
sh_names=[]
for idx in range(len(sh_columns)):
    if sh_columns[idx][0] in sh_select_columns:
        sh_schema[sh_columns[idx][0]] = sh_columns[idx][1]
        sh_use_cols.append(idx)
        sh_names.append(sh_columns[idx][0])
            
sh_df = pd.read_csv("/tmp/20181016.txt",
                    usecols=sh_use_cols,
                    names=sh_names,
                    dtype=sh_schema,
                    header=None,
                    delimiter="\t",
                    date_parser=pd.to_datetime,
                    parse_dates=["pos_time"])<jupyter_output><empty_output><jupyter_text>根据经纬度信息，构造位置点信息：<jupyter_code>sh_df["pos_point"]=GeoSeries.point(sh_df.pos_longitude,sh_df.pos_latitude)
sh_df<jupyter_output><empty_output><jupyter_text>我们可以根据某一辆车的车牌号，来还原出该车辆的运行轨迹，首先我们选择其中一个车牌号，并筛选该车牌号的所有数据：<jupyter_code>one_trunck_plate_number=sh_df.plate_number[0]
print(one_trunck_plate_number)
one_truck_df = sh_df[sh_df.plate_number==one_trunck_plate_number]<jupyter_output>沪DK7362<jupyter_text>在地图上绘制这辆车的所有轨迹点：<jupyter_code>from keplergl import KeplerGl
KeplerGl(data={"car_pos": pd.DataFrame(data={'car_pos':one_truck_df.pos_point.to_wkt()})})<jupyter_output>User Guide: https://docs.kepler.gl/docs/keplergl-jupyter<jupyter_text>接下来，我们将根据上海市的路网信息看上述车辆的运行轨迹，首先加载路网信息：<jupyter_code>sh_roads=pd.read_csv("/tmp/sh_roads.csv", 
                     dtype={"roads":"string"},
                     usecols=[0],
                     names=["roads"],
                     header=None,
                     delimiter='|')
sh_roads=GeoSeries(sh_roads.roads)
sh_roads<jupyter_output><empty_output><jupyter_text>同时在地图上绘制上述轨迹点信息和路网信息：<jupyter_code>one_truck_roads=KeplerGl(data={"car_pos": pd.DataFrame(data={'car_pos':one_truck_df.pos_point.to_wkt()})})
one_truck_roads.add_data(data=pd.DataFrame(data={'sh_roads':sh_roads.to_wkt()}),name="sh_roads")
one_truck_roads<jupyter_output>User Guide: https://docs.kepler.gl/docs/keplergl-jupyter<jupyter_text>局部放大后可以发现有些轨迹点并不在路上。 我们认为并不在路上的轨迹点为噪点信息，需要将轨迹点绑定到离它最近的一条道路上：<jupyter_code>is_near_road=arctern.near_road(sh_roads,sh_df.pos_point)
sh_near_road_df=sh_df[is_near_road]
on_road=arctern.nearest_location_on_road(sh_roads, sh_near_road_df.pos_point)
on_road=GeoSeries(on_road)
on_road<jupyter_output><empty_output><jupyter_text>使用绑定到路上点，重新构造 `DataFrame`：<jupyter_code>sh_on_road_df=pd.DataFrame(data={"plate_number":sh_near_road_df.plate_number,
                                 "pos_time":sh_near_road_df.pos_time,
                                 "on_road":on_road
                                })
sh_on_road_df<jupyter_output><empty_output><jupyter_text>再次绘制前述车辆的轨迹点及上海市的路网信息：<jupyter_code>one_on_road_df=sh_on_road_df[sh_on_road_df.plate_number==one_trunck_plate_number]
one_on_roads=KeplerGl(data={"car_pos": pd.DataFrame(data={'car_pos':one_on_road_df.on_road.to_wkt()})})
one_on_roads.add_data(data=pd.DataFrame(data={'sh_roads':sh_roads.to_wkt()}),name="sh_roads")
one_on_roads<jupyter_output>User Guide: https://docs.kepler.gl/docs/keplergl-jupyter<jupyter_text>局部放大后，可以发现所有点均在道路上  分析 我们了解到，上海市的道路网信息共 74693 条记录，但并不是所有道路都会被渣土车经过，我们将对渣土车经过的道路进行分析，查看哪些道路是渣土车经过频率最高的。首先筛选出渣土车经过的所有道路：<jupyter_code>all_roads=arctern.nearest_road(sh_roads,sh_on_road_df.on_road)
all_roads=GeoSeries(all_roads)
road_codes, road_uniques = pd.factorize(all_roads)<jupyter_output><empty_output><jupyter_text>打印所有渣土车经过的道路数据，及所占整体道路的百分比：<jupyter_code>print(len(road_uniques))
print(len(road_uniques)*100.0/len(sh_roads))<jupyter_output>16450
22.0234827895519<jupyter_text>绘制所有渣土车经过的道路：<jupyter_code>KeplerGl(data={"all_roads": pd.DataFrame(data={'all_roads':GeoSeries(road_uniques).to_wkt()})})<jupyter_output>User Guide: https://docs.kepler.gl/docs/keplergl-jupyter<jupyter_text>可以看出，对于一些主干道路，每辆渣土车都可能会经过，导致这条道路上的 `GPS` 信号采样点比较多，同时因为是主干道路，经过车辆比较多，导致渣土车速度缓慢，进一步加强该道路上的 `GPS` 采样点数据。 接下来我们将根据道路上渣土车的采样点数目，来寻找较繁忙的道路。统计每条道路上的 `GPS` 采样点数目，并重新构建 `DataFrame` ，我们把道路上 `GPS` 采样点的数据记为道路权重：<jupyter_code>roads_codes_series = pd.Series(road_codes)
roads_codes_series = roads_codes_series.value_counts()
roads_codes_series = roads_codes_series.sort_index()

sh_road_weight = pd.DataFrame(data={"on_road":GeoSeries(road_uniques),
                                    "weight_value":roads_codes_series
                                   })

sh_road_weight<jupyter_output><empty_output><jupyter_text>输出道道路权重的基本概况：<jupyter_code>sh_road_weight.weight_value.describe()<jupyter_output><empty_output><jupyter_text>可以发现大部分道路是不繁忙的，但是也存在特别繁忙的道路。将`weight_value`绘制成柱状图，可进一步佐证：<jupyter_code>import matplotlib.pyplot as plt
plt.bar(sh_road_weight.index,sh_road_weight.weight_value)
plt.show()<jupyter_output><empty_output><jupyter_text>按照道路权重，对所有道路做排序：<jupyter_code>sh_sorted_road=sh_road_weight.sort_values(by=['weight_value'],ascending=False)
sh_sorted_road<jupyter_output><empty_output><jupyter_text>选取最繁忙的前100个路段：<jupyter_code>sh_sorted_road.iloc[0:100]<jupyter_output><empty_output><jupyter_text>在地图上绘制最繁忙的前100个路段：<jupyter_code>KeplerGl(data={"on_roads": pd.DataFrame(data={'on_roads':sh_sorted_road.on_road.iloc[0:100].to_wkt()})})<jupyter_output>User Guide: https://docs.kepler.gl/docs/keplergl-jupyter
StarcoderdataJupyterStructuredCleanDedup