- 博客(349)
- 收藏
- 关注
RSS订阅
原创 scala小知识
计算对象在内存的大小ObjectSizeCalculatorscala> import jdk.nashorn.internal.ir.debug.ObjectSizeCalculator;import jdk.nashorn.internal.ir.debug.ObjectSizeCalculatorscala> ObjectSizeCalculator.getObjectSize(Integer.valueOf(122))val res7: Long = 16scala>
2022-02-11 15:49:55
470
原创 java.util.concurrent.Executors详解
Class Executors APIFactory and utility methods for Executor, ExecutorService, ScheduledExecutorService, ThreadFactory, and Callable classes defined in this package. This class supports the following kinds of methods:Methods that create and return an Ex
2022-04-25 11:24:12
1091
原创 JDK8 日期时间相关 java.time.*
LocalDateTime, LocalTime,LocalDate 区别LocalDate：不包含具体时间的日期 2022-04-15LocalTime：不含日期的时间 16:11:45.864LocalDateTime：包含了日期及时间 2022-04-15T16:11:18.292scala> import java.time.{LocalDateTime, LocalTime,LocalDate,Duration, Instant}import java.time.{LocalD
2022-04-15 16:31:34
2349
原创 Scala Either Left Right
Either官网Either代表两个可能类型的值，两个之间没有交集是scala.util.Left or scala.util.Right 的实现类可以做scala.option的替换 ，scala.util.Left 替换scala.None,scala.util.Right替换scala.Some时实践中使用scala.util.Left作为Failurescala.util.Right作为成功的值官方demoobject EitherDemo { def main(args:
2022-02-10 14:45:12
449
原创 java.util.concurrent.ConcurrentHashMap 详解
ConcurrentHashMapConcurrentHashMap使用的是分段锁技术,将ConcurrentHashMap将锁一段一段的存储，然后给每一段数据配一把锁（segment），当一个线程占用一把锁（segment）访问其中一段数据的时候，其他段的数据也能被其它的线程访问，默认分配16个segment。默认比Hashtable效率提高16倍。ConcurrentHashMap是Map的一种并发实现，在该类中元素的read操作都是无锁了，而write操作需要被同步。这非常适合于读操作远大于写操作
2022-02-08 13:19:58
1659
原创 java.util.Properties
java.util.Properties类import java.io.{FileInputStream, FileOutputStream}import java.util.Propertiesobject Properties { def main(args: Array[String]): Unit = { val props: Properties =new Properties() val stream: FileInputStream =new FileIn
2022-02-07 15:56:52
2530
原创 Scala System详解
SystemSystem 类位于 java.lang 包，代表当前 Java 程序的运行平台，系统级的很多属性和控制方法都放置在该类的内部。由于该类的构造方法是 private 的，所以无法创建该类的对象，也就是无法实例化该类。scala> System.arraycopy( getenv( runFinalization()clearProperty(
2022-02-03 15:34:16
1573
原创 scala stringUtils 详解
引用import org.apache.commons.lang3.stringUtilsscala> val s1="1234567890"s1: String = 1234567890scala> val s2="abcdefghij"s2: String = abcdefghijascala> StringUtils.aabbreviate abbreviateMiddle appendIfMissing appendIfMissingIgnoreC
2022-01-29 10:46:09
1817
原创 scala 集合List Array Seq详解
scala> val l=List(1 to 10:_*)val l: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)scala> val a=Array(1 to 10:_*)val a: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)scala> val s=Seq(1 to 10:_*)val s: Seq[Int] = List(1, 2, 3, 4, 5, 6, 7,
2022-01-25 17:26:10
1337
原创 scala spark df 动态过滤fliter
参考：how to filter out a null value from spark dataframescala> df.show()+----+---+--------+| age| id| name|+----+---+--------+|null| 1| Michael|| 30| 1| Tom|| 19| 2| Justin|+----+---+--------+每列不为null val filterCond: Column = df.
2022-01-13 16:53:52
1766
原创 Java System.exit()
System.exit(0) ： 将整个虚拟机里的内容都关掉，内存都释放掉！正常退出程序。System.exit(1) ： 非正常退出程序System.exit(-1) ：非正常退出程序
2021-12-07 18:44:06
750
原创 hadoop hdfs 文件操作
查看snappy 压缩文件hadoop fs -D io.compression.codec.snappy.native=true -text hdfs://haruna/user/tiger/test/****/j211028113306ga9NT6000891/2021-09-30.snappy
2021-10-28 14:17:35
767
原创 java.lang.RuntimeException: Cannot reserve additional contiguous bytes in the vectorized reader
spark任务报错Job aborted due to stage failure: Task 2049 in stage 515.0 failed 5 times, most recent failure: Lost task 2049.3 in stage 515.0 (TID 241301, n11-147-025.byted.org, executor 1078): java.lang.RuntimeException: Cannot reserve additional contiguous
2021-10-19 20:40:59
1453
原创 HIVE map
map 构造函数map(key1, value1, key2, value2, …) Creates a map with the given key/value pairsdemoselect map("a", 1, "b", 2, "c", "dd") as m1, map("a", 1, "b", 2, "c", 3) as m2, map(1, 1, 2, 2, 3, 3) as m2返回m1 map<string,string>m2 map<s
2021-10-18 20:03:10
552
原创 HIVE explode
作用explode() takes in an array (or a map) as an input and outputs the elements of the array (map) as separate rows. UDTFs can be used in the SELECT expression list and as a part of LATERAL VIEW.explode() 接收一个 array 或 map 类型的数据作为输入，然后将 array 或 map 里面的元素按照每
2021-10-18 18:06:56
199
原创 HIVE nvl 空值转换函数
作用空值转换函数nvl(value,default_value) - Returns default value if value is null else returns valuedemoselect nvl("abc", "123") as n1, nvl("", "123") as n2, nvl(null, "123") as n2输出abc123
2021-10-18 17:57:48
1298
原创 HIVE str_to_map将字符串转为map格式
作用将字符串转为map格式map<string,string>str_to_map(text[, delimiter1, delimiter2])Splits text into key-value pairs using two delimiters. Delimiter1 separates text into K-V pairs, and Delimiter2 splits each K-V pair. Default delimiters are ‘,’ f
2021-10-18 17:52:12
591
原创 hive not in效率优化
hive not in 效率忍不了select shop_idfrom whitelistwhere date = '${date-1}' and shop_id not in ( select shop_id from whitelist where date = '${date-2}' ) limit 20愣是跑了十几分钟没完修改后select a.shop_idfrom ( selec
2021-09-29 11:35:23
965
原创 Linux安装Anaconda
下载Anaconda官方地址清华大学找到所需要版本下载 wget https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/Anaconda3-2020.11-Linux-x86_64.shbash Anaconda3-2020.11-Linux-x86_64.sh一路enter输入yes问题AttributeError: module 'importlib._bootstrap_external' has no attribu
2021-07-28 10:52:56
1271
原创 Unable to attach or mount volumes: unmounted volumes=[flannel-cfg], unattached volumes=[flannel-cfg
describe pod参考# cni0、flannel.1 刪除他，砍了他，踢了他# 先移除 flannelkubectl delete daemonsets -n kube-system -l app=flannelsudo ip link delete cni.0sudo ip link delete flannel.1#
2021-07-23 15:12:29
7250
原创 K8S dashboard 无法访问
背景node 正常pod 正常但是dashboard 无法访问root@n226-060-152:/etc/init.d# kubectl get pod --all-namespacesNAMESPACE NAME READY STATUS RESTARTS AGEkube-system coredns-66bff467f8-nczr6
2021-07-22 19:57:59
3479
3
原创 K8S node节点not ready
部署K8snode 节点not readyroot@n226-060-152:/opt/cni/bin# kubectl get nodeNAME STATUS ROLES AGE VERSIONn224-214-218 NotReady <none> 119s v1.18.4n226-060-152 Ready master 21h v1.18.4n226-070-127 NotReady &l
2021-07-22 15:33:13
5566
原创 Failed to list *v1.Service: Get https://10.96.0.1:443/api/v1/services?limit=500&resourceVersion=0: d
kubectl get pods --all-namespaceskubectl describe pod coredns-66bff467f8-kx4ck -n kube-systemkubectl logs -f coredns-66bff467f8-kx4ck -n kube-systemlinux/amd64, go1.13.6, da7f65bE0713 08:17:10.668228 1 reflector.go:153] pkg/mod/k8s.io/client-.
2021-07-13 16:29:39
8459
2
原创 Error: could not find a ready tiller pod
helm version 出错lynx@n149-136-019:/root$ helm versionClient: &version.Version{SemVer:"v2.16.9", GitCommit:"8ad7037828e5a0fca1009dabe290130da6368e39", GitTreeState:"clean"}Error: could not find a ready tiller podkubectl delete deployment tiller-depl
2021-07-13 16:13:51
137
原创 helm install Error: timed out waiting for the condition
背景helm 安装jhub时超时lynx@n149-136-019:~/magellan_docker/magellan_k8s/k8s_dev/k8s_jhub$ helm upgrade --install jhub jupyterhub/jupyterhub \> --namespace jhub \> --version=0.9.0 \> --values k8s_config.yamlRelease "jhub" does not exist. Inst
2021-07-13 15:57:59
8833
原创 configmaps is forbidden: User “system:serviceaccount:kube-system:default“ cannot list resource “con
helm list时报错lynx@n149-136-019:~/magellan_docker/magellan_k8s/k8s_dev/k8s_jhub$ helm listError: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list resource "configmaps" in API group "" in the namespace "kube-system"so
2021-07-13 15:27:20
4235
原创 K8S卸载并且重装的心酸历程
背景K8S证书过期了，然后不小心删除了个文件，然后就悲剧了报错总结root@n149-136-019:~# kubectl get nodeerror: no configuration has been provided, try setting KUBERNETES_MASTER environment variable卸载之后，重装sudo kubeadm init --kubernetes-version v1.18.14 --pod-network-cidr=10.244.0.0/1
2021-07-13 13:36:38
2007
原创 Unable to connect to the server: x509: certificate has expired or is not yet valid
背景K8S 集群部署的服务出现问题("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')])",)尝试进入 K8S主节点重启服务报错Unable to connect to the server: x509: certificate has expired or is not yet validUnable to connect to t
2021-07-12 17:49:04
890
原创 antd table 默认排序不起作用
参考官网划重点：columns 中定义了 filteredValue 和 sortOrder 属性即视为受控模式。只支持同时对一列进行排序，请保证只有一列的 sortOrder 属性是生效的。务必指定 column.key。sortOrder: sortedInfo.columnKey === ‘name’ && sortedInfo.order,初始化 sortedInfoconst [sortedInfo, setSortedInfo] = useStat
2021-07-09 10:53:00
3220
原创 mybatis if 并且判断列表是否为空
<select id="getres" parameterType="java.util.List" resultType="……RiskFeature"> select * from f where g.is_deleted=0 <if test="items != null and items.size()>0"> and f.s.
2021-05-25 16:06:13
2515
原创 network: failed to set bridge addr: “cni0“ already has an IP address different from 10.244.2.1/2
错误信息042机器Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "fa4d75e60f45f3da456587f5be42a98b0019e2f206b54088cf93f6caf29b7310" network for pod "hook-image-puller-zx9vl": networkPlugin cni failed to set up..
2021-05-20 15:04:43
370
原创 K8S常用命令
kubectl Cheat Sheet查看查看 某个 ns或者全部的podkubectl get pod -n jhubkubectl get pods --all-namespaceskubectl get nodeskubectl describe pod kube-proxy-sstwk -n kube-system部署K8S时需要的镜像sudo kubeadm config images list查看端口占用kubectl get svc -n kub
2021-05-20 14:18:05
139
原创 Spring MVC笔记
五大组件前端控制器 DispatcherServlet映射处理器 HandlerMapping处理器 Controller模型和视图 ModelAndView视图解析器 ViewResovler运行原理
2021-04-23 14:57:32
59
空空如也
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人